Multi-mode parallel graphics rendering system (MMPGRS) embodied within a host computing system and employing the profiling of scenes in graphics-based applications

ABSTRACT

Multi-mode parallel graphics rendering system (MMGRPS) embodied within a host computing system and employing the profiling of scenes in a graphics-based application. The MMPGRS supports multiple modes of parallel operation selected from the group consisting of object division, image division, and time division. A plurality of graphic processing pipelines (GPPLs) support a parallel graphics rendering process that employs one or more of the object division, image division and/or time division modes of parallel operation in order to execute graphic commands and process graphics data, and render pixel-composited images containing graphics for display on a display device during the run-time of the graphics-based application. An automatic mode control module automatically controls the mode of parallel operation of the MMPGRS during the run-time of the graphics-based application.

CROSS-REFERENCE TO RELATED CASES

The present application is a Continuation of U.S. application Ser. No.11/897,536 filed Aug. 30, 2007; which is a Continuation-in-Part (CIP) ofthe following Applications: U.S. application Ser. No. 11/789,039 filedApr. 23, 2007; U.S. application Ser. No. 11/655,735 filed Jan. 18, 2007,which is based on Provisional Application Ser. No. 60/759,608 filed Jan.18, 2006; U.S. application Ser. No. 11/648,160 filed Dec. 31, 2006; U.S.application Ser. No. 11/386,454 filed Mar. 22, 2006; U.S. applicationSer. No. 11/340,402 filed Jan. 25, 2006, which is based on ProvisionalApplication No. 60/647,146 filed Jan. 25, 2005; U.S. application Ser.No. 10/579,682 filed May 17, 2006, which is a National Stage Entry ofInternational Application No. PCT/IL2004/001069 filed Nov. 19, 2004,which is based on Provisional Application Ser. No. 60/523,084 filed Nov.19, 2003; each said patent application being commonly owned by LucidInformation Technology, Ltd., and being incorporated herein by referenceas if set forth fully herein.

BACKGROUND OF INVENTION

1. Field of Invention

The present invention relates generally to the field of computergraphics rendering, and more particularly, ways of and means forimproving the performance of parallel graphics rendering processessupported on multiple 3D graphics processing pipeline (GPPL) platformsassociated with diverse types of computing machinery, including, but notlimited, to PC-level computers, game console systems,graphics-supporting application servers, and the like.

2. Brief Description of the State of Knowledge in the Art

There is a great demand for high performance three-dimensional (3D)computer graphics systems in the fields of product design, simulation,virtual-reality, video-gaming, scientific research, and personalcomputing (PC). Clearly a major goal of the computer graphics industryis to realize real-time photo-realistic 3D imagery on PC-basedworkstations, desktops, laptops, and mobile computing devices. Ingeneral, there are two fundamentally different classes of machines inthe 3D computer graphics field, namely: (1) Object-Oriented GraphicsSystems, wherein 3D scenes are represented as a complex of geometricobjects (primitives) in 3D continuous geometric space, and 2D views orimages of such 3D scenes are computed using geometrical projection, raytracing, and light scattering/reflection/absorption modeling techniques,typically based upon laws of physics; and (2) VOlume ELement (VOXEL)Graphics Systems, wherein 3D scenes and objects are represented as acomplex of voxels (x,y,z volume elements) represented in 3D CartesianSpace, and 2D views or images of such 3D voxel-based scenes are alsocomputed using geometrical projection, ray tracing, and lightscattering/reflection/absorption modeling techniques, again typicallybased upon laws of physics. Examples of early GDL-based graphics systemsare disclosed in U.S. Pat. No. 4,862,155, whereas examples of earlyvoxel-based 3D graphics systems are disclosed in U.S. Pat. No.4,985,856, each incorporated herein by reference in its entirety. In thecontemporary period, most PC-based computing systems include a 3Dgraphics subsystem based the “Object-Orient Graphics” system design. Insuch graphics system design, “objects” within a 3D scene are representedby 3D geometrical models, and these geometrical models are typicallyconstructed from continuous-type 3D geometric representations including,for example, 3D straight line segments, planar polygons, polyhedra,cubic polynomial curves, surfaces, volumes, circles, and quadraticobjects such as spheres, cones, and cylinders (i.e. geometrical data andcommands). These 3D geometrical representations are used to modelvarious parts of the 3D scene or object, and are expressed in the formof mathematical functions evaluated over particular values ofcoordinates in continuous Cartesian space. Typically, the 3D geometricalrepresentations of the 3D geometric model are stored in the format of agraphical display list (i.e. a structured collection of 2D and 3Dgeometric primitives). Currently, planar polygons, mathematicallydescribed by a set of vertices, are the most popular form of 3Dgeometric representation.

Once modeled using continuous 3D geometrical representations, the 3Dscene is graphically displayed (as a 2D view of the 3D geometricalmodel) along a particular viewing direction, by repeatedlyscan-converting the stream of graphics commands and data (GCAD). At thecurrent state of the art, the scan-conversion process can be viewed as a“computational geometry” process which involves the use of (i) ageometry processor (i.e. geometry processing subsystem or engine) aswell as a pixel processor (i.e. pixel processing subsystem or engine)which together transform (i.e. project, shade and color) the graphicsobjects and bit-mapped textures, respectively, into an unstructuredmatrix of pixels. The composed set of pixel data is stored within a 2Dframe buffer (i.e. Z buffer) before being transmitted to and displayedon the surface of a display screen.

A video processor/engine refreshes the display screen using the pixeldata stored in the 2D frame buffer. Any changes in the 3D scene requiresthat the geometry and pixel processors repeat the wholecomputationally-intensive pixel-generation pipeline process, again andagain, to meet the requirements of the graphics application at hand. Forevery small change or modification in viewing direction of the humansystem user, the graphical display list must be manipulated andrepeatedly scan-converted. This, in turn, causes both computational andbuffer contention challenges which slow down the working rate of thegraphics system. To accelerate this computationally-intensive graphicsprocessing pipeline process, custom hardware including geometry, pixeland video engines, have been developed and incorporated into mostconventional graphics system designs.

In order to render a 3D scene (from its underlying graphics commands anddata) and produce high-resolution graphical projections for display on adisplay device, such as a LCD panel, early 3D graphics systems attemptedto relieve the host CPU of computational loading by employing a singlegraphics pipeline comprising a single graphics processing unit (GPU),supported by video memory.

As shown in FIGS. 1A1, 1A2 and 1A3, a typical PC based graphicarchitecture has an external graphics card 105 comprising a graphicsprocessing unit (GPU) and video memory. As shown, the graphic card isconnected to the display 106 on one side, and the CPU 101 through bus(e.g. PCI-Express) 107 and Memory Bridge 103 (termed also “chipset”,e.g. 975 by Intel), on the other side. As shown in FIG. 1A 3, the hostCPU program/memory space stores the graphics applications, the standardgraphics library, and the vendor's GPU drivers.

As shown in FIGS. 1B1, 1B2 and 1B3, a typical prior art PC-basedcomputing system employs a conventional graphics architecture employinga North memory bridge with an integrated graphics device (IGD) 103. TheIGD supports a single graphics pipeline process, and is operably coupledto a South bridge, via a PCI-express bus, for supporting theinput/output ports of the system. As shown, the IGD includes a videoengine, a 2D engine, a 3D engine, and a display engine.

As shown in FIG. 1B 4, a prior art PC-based computing system employs aconventional Fusion-type CPU/GPU hybrid architecture, wherein a singleGPU implemented on the same die as the CPU is used to support a graphicspipeline that drives an external display device. As shown, themotherboard supports the processor die, memory, a bridge with a displayinterface for connecting to a display device 106, and a PCI-express bus.As shown, the processor die supports a CPU 1241, a GPU 1242, L2 cache,buffers, an Interconnect (e.g. crossbar switch), a hyper transportmechanism and a memory controller.

As shown in FIG. 1C, the process of rendering three successive frames bya single GPU is graphically illustrated. Notably, this graphicalrendering process may be supported using any of the single GPU-basedcomputing systems described above. During operation, the application,assisted by the graphics library, creates a stream of graphics commandsand data describing a 3D scene. The stream is then pipelined through theGPU's geometry and pixel subsystems so as to create a bitmap of pixelsin the Frame Buffer, and finally a rendered image of the scene isdisplayed on a display screen. The generation of a sequence ofsuccessive frames produces a visual illusion of a dynamic picture.

While the performance of single-GPU powered computing systems havegreatly improved in As shown in FIG. 1B 5, the structure of a GPUsubsystem 124 on a graphics card or in an IGD comprises: a video memorywhich is external to GPU, and two 3D engines: (i) a transform boundgeometry subsystem 224 for processing 3D graphics primitives; (ii) and afill bound pixel subsystem 225. The video memory shares its storageresources among geometry buffer 222 through which all geometric (i.e.polygonal) data is transferred, commands buffer, texture buffers 223,and Frame Buffer 226.

Limitations of a single graphics pipeline arise from its typicalbottlenecks. The first potential bottleneck 221 stems from transferringdata from CPU to GPU. Two other bottlenecks are video memory related:geometry data memory limits 222, and texture data memory limits 223.There are two additional bottlenecks inside the GPU: transform bound 224in the geometry subsystem, and fragment rendering 225 in pixelsubsystem. These bottlenecks determine overall throughput. In general,the bottlenecks vary over the course of a graphics application.

In high-performance graphics applications, the number of computationsrequired to render a 3D scene and produce high-resolution graphicalprojections, greatly exceeds the capabilities of systems employing asingle GPU graphics subsystem. Consequently, the use of parallelgraphics pipelines, and multiple graphics processing units (GPUs), havebecome the rule for high-performance graphics system architecture anddesign, in order to relieve the overload presented by the differentbottlenecks associated with single GPU graphics subsystems.

In FIG. 2A, there is shown an advanced chipset (e.g. Bearlake by Intel)having two buses 107, 108 instead of one, and allowing theinterconnection of two external graphics cards in parallel: primary card105 and secondary card 104, to share the computation load associatedwith the 3D graphics rendering process. As shown, the display 106 isattached to the primary card 105. It is anticipated that even moreadvanced commercial chipsets with greater than two buses will appear inthe future, allowing the interconnection of more than two graphic cards.

As shown in FIG. 2B, the general software architecture of prior artgraphic system 200 comprises: the graphics application 201, standardgraphics library 202, and the vendor's GPU drivers (203). This graphicsoftware environment resides in the “program space” of main memory 102on the host computer system. As shown, the graphic application 201 runsin the program space (i.e. memory space), building up the 3D scene,typically as a data base of polygons, where each polygon is representedas a set of vertices. The vertices and others components of thesepolygons are transferred to the graphic card(s) for rendering, anddisplayed as a 2D image, on the display screen.

In FIG. 2C, the structure of a GPU subsystem on the graphics card isshown comprising: a video memory disposed external to the GPU, and two3D engines: (i) a transform bound geometry subsystem 224 for processing3D graphics primitives; and (ii) a fill bound pixel subsystem 225. Thevideo memory shares its storage resources among geometry buffer 222,through which all geometric (i.e. polygonal) data is transferred to thecommands buffer, texture buffers 223, and Frame Buffer FB 226.

As shown in FIG. 2C, the division of graphics data among GPUs reduces(i) the bottleneck 222 posed by the video memory footprint at each GPU,(ii) the transform bound processing bottleneck 224, and (iii) the fillbound processing bottleneck 225.

However, when using a multiple GPU graphics architecture of the typeshown in FIGS. 2A through 2C, there is a need to distribute thecomputational workload associated with interactive parallel graphicsrendering processes. To achieve this objective, two different kind ofparallel rendering methods have been applied to PC-based dual GPUgraphics systems of the kind illustrated in FIGS. 2A through 2C, namely:the Time Division Method of Parallel Graphics Rendering illustrated inFIG. 2D; and the Image Division Method of Parallel Graphics Renderingillustrated in FIG. 2E.

Notably, a third type of method of parallel graphics rendering, referredto as the Object Division Method, has been developed over the years andpracticed exclusively on complex computing platforms requiring complexand expensive hardware platforms for compositing the pixel output of themultiple graphics processing pipelines (GPPLs). The Object DivisionMethod, illustrated in FIG. 3A, can be found applied on conventionalgraphics platforms of the kind shown in FIG. 3, as well as onspecialized graphics computing platforms as described in US PatentApplication Publication No. US 2002/0015055, assigned to SiliconGraphics, Inc. (SGI), published on Feb. 7, 2002, and incorporated hereinby reference.

While the differences between the Image, Frame and Object DivisionMethods of Parallel Graphics Rendering will be described below, it willbe helpful to first briefly describe the five (5) basic stages or phasesof the parallel graphics rendering process, which all three such methodsof parallel rendering have in common, namely:

(1) the Decomposition Phase, wherein the 3D scene or object is analyzedand its corresponding graphics display list data and commands areassigned to particular graphics pipelines available on the parallelmultiple GPU-based graphics platform;

(2) the Distribution Phase, wherein the graphics data and commands aredistributed to particular available graphics processing pipelinesdetermined during the Decomposition Phase;

(3) the Rendering Phase, wherein the geometry processingsubsystem/engine and the pixel processing subsystem/engine along eachgraphics processing pipeline of the parallel graphics platform uses thegraphics data and commands distributed to its pipeline, and transforms(i.e. projects, shades and colors) the graphics objects and bit-mappedtextures into a subset of unstructured matrix of pixels;

(4) the Recomposition Phase, wherein the parallel graphics platform usesthe multiple sets of pixel data generated by each graphics pipeline tosynthesize (or compose) a final set of pixels that are representative ofthe 3D scene (taken along the specified viewing direction), and thisfinal set of pixel data is then stored in a frame buffer (FB); and

(5) the Display Phase, wherein the final set of pixel data retrievedfrom the frame buffer, and provided to the screen of the device deviceof the system.

As will be explained below with reference to FIGS. 3B through 3D, eachof these three different methods of parallel graphics rendering has bothadvantages and disadvantages.

Image Division Method of Parallel Graphics Rendering

As illustrated in FIG. 2D, the Image Division (Sort-First) Method ofParallel Graphics Rendering distributes all graphics display list dataand commands to each of the graphics pipelines, and decomposes the finalview (i.e. projected 2D image) in Screen Space, so that, each graphicalcontributor (e.g. graphics pipeline and GPU) renders a 2D tile of thefinal view. This mode has a limited scalability due to the paralleloverhead caused by objects rendered on multiple tiles. There are twoimage domain modes, all well known in prior art. They differ by the waythe final image is divided among GPUs.

(1) The Split Frame Rendering mode divides up the screen among GPUs bycontinuous segments. e.g. two GPUs each one handles about one half ofthe screen. The exact division may change dynamically due to changingload across the screen image. This method is used in nVidia's SLI™multiple-GPU graphics product.

(2) Tiled Frame Rendering mode divides up the image into small tiles.Each GPU is assigned tiles that are spread out across the screen,contributing to good load balancing. This method is implemented by ATI'sCrossfire™ multiple GPU graphics card solution.

In image division, the entire database is broadcast to each GPU forgeometric processing. However, the processing load at each PixelSubsystem is reduced to about 1/N. This way of parallelism relieves thefill bound bottleneck 225. Thus, the image division method ideally suitsgraphics applications requiring intensive pixel processing.

Time Division (DPlex) Method of Parallel Graphics Rendering

As illustrated in FIG. 2F, the Time Division (DPlex) Method of ParallelGraphics Rendering distributes all display list graphics data andcommands associated with a first scene to the first graphics pipeline,and all graphics display list data and commands associated with asecond/subsequent scene to the second graphics pipeline, so that eachgraphics pipeline (and its individual rendering node or GPU) handles theprocessing of a full, alternating image frame. Notably, while thismethod scales very well, the latency between user input and finaldisplay increases with scale, which is often irritating for the user.Each GPU is give extra time of N time frames (for N parallel GPUs) toprocess a frame. Referring to FIG. 3, the released bottlenecks are thoseof transform bound 224 at geometry subsystem, and fill bound 225 atpixel subsystem. Though, with large data sets, each GPU must access allof the data. This requires either maintaining multiple copies of largedata sets or creating possible access conflicts to the source copy atthe host swelling up the video memory bottlenecks 222, 223 and datatransfer bottleneck 221.

Object Division (Sort-Last) Method of Parallel Graphics Rendering

As illustrated in FIG. 3B, the Object Division (Sort-Last) Method ofParallel Graphics Rendering decomposes the 3D scene (i.e. rendereddatabase) and distributes graphics display list data and commandsassociated with a portion of the scene to the particular graphicspipeline (i.e. rendering unit), and recombines the partially renderedpixel frames, during recomposition. The geometric database is thereforeshared among GPUs, reducing the load on the geometry buffer, thegeometry subsystem, and even to some extent, the pixel subsystem. Themain concern is how to divide the data in order to keep load balance. Anexemplary multiple-GPU platform of FIG. 3B for supporting theobject-division method is shown in FIG. 3A. The platform requirescomplex and costly pixel compositing hardware which prevents its currentapplication in a modern PC-based computer architecture.

Today, real-time graphics applications, such as advanced video games,are more demanding than ever, utilizing massive textures, abundance ofpolygons, high depth-complexity, anti-aliasing, multi-pass rendering,etc., with such robustness growing exponentially over time.

Conventional PC-level dual-mode parallel graphics systems employingmultiple-GPUs, such as nVidia's SLI™ multiple-GPU graphics platform,support either the Time Division Mode (termed Alternate Frame Rendering)of parallelism, or the Image Division Mode (termed Split FrameRendering) of parallelism, which is automatically selected duringapplication set-up (e.g. by the vendor's driver). However, once agraphics-based application is set-up and the time or image division modeof parallel operation selected, the selected mode of parallel operationis fixed during application run-time.

Clearly, conventional PC-based graphics systems fail to address thedynamically changing needs of modern graphics applications. By theirvery nature, prior art PC-based graphics systems are unable to resolvethe variety of bottlenecks (e.g. geometry limited, pixel limited, datatransfer limited, and memory limited) summarized in FIG. 3C 1, thatdynamically arise along 3D graphic pipelines. Consequently, such priorart graphics systems are often unable to maintain a high and steadylevel of performance throughout a particular graphics application.

Indeed, a given graphics processing pipeline along a parallel graphicsrendering system is only as strong as the weakest link of it stages, andthus a single bottleneck determines the overall throughput along thegraphics pipelines, resulting in unstable frame-rate, poor scalability,and poor performance.

And while each parallelization mode described above and summarized inFIG. 3C 2 solves only part of the bottleneck dilemma currently existingalong the PC-based graphics pipelines, no one parallelization method, inand of itself, is sufficient to resolve all bottlenecks in demandinggraphics applications, and enable quantum leaps in graphics performancenecessary for photo-realistic imagery demanded in real-time interactivegraphics environments.

Thus, there is a great need in the art for a new and improved way of andmeans for practicing parallel 3D graphics rendering processes in modernmultiple-GPU based computer graphics systems, while avoiding theshortcomings and drawbacks of such prior art methodologies andapparatus.

SUMMARY AND OBJECTS OF THE PRESENT INVENTION

Accordingly, a primary object of the present invention is to provide anew and improved method of and apparatus for practicing parallel 3Dgraphics rendering processes in modern multiple-GPU based computergraphics systems, while avoiding the shortcomings and drawbacksassociated with prior art apparatus and methodologies.

Another object of the present invention is to provide a novel multi-modeparallel graphics rendering system (MMPGRS) embodied within a hostcomputing system having (i) host memory space (HMS) for storing one ormore graphics-based applications and a graphics library for generatinggraphics commands and data (GCAD) during the run-time (i.e. execution)of the graphics-based application, (ii) one or more CPUs for executingsaid graphics-based applications, and (iii) a display device fordisplaying images containing graphics during the execution of saidgraphics-based applications.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering and display system comprising: a multi-modeparallel graphics rendering subsystem supporting multiple modes ofparallel operation selected from the group consisting of objectdivision, image division, and time division; a plurality of graphicprocessing pipelines (GPPLs) supporting a parallel graphics renderingprocess that employs one of the object division, image division and/ortime division modes of parallel operation in order to execute graphiccommands and process graphics data (GCAD) render pixel-composited imagescontaining graphics for display on a display device during the run-timeof the graphics-based application; and an automatic mode control module(AMCM) for automatically controlling the mode of parallel operationduring the run-time of the graphics-based application.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering and display system, wherein the automaticmode control module employs the profiling of scenes in saidgraphics-based application.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering and display system, wherein the automaticmode control module employs the profiling of scenes in thegraphics-based application, on an image frame by image frame basis.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering and display system, wherein the profiling ofscenes in the graphics-based application, is carried out in real-time,during run-time of the graphics-based application, on an image frame byimage frame basis.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering and display system, wherein said real-timeprofiling of scenes in the graphics-based application involves (i)collecting and analyzing performance data associated with the MMPGRS andthe host computing system, during application run-time, (ii)constructing scene profiles for the image frames associated withparticular scenes in the particular graphics-based application, and(iii) maintaining the scene profiles in a application/scene profiledatabase that is accessible to the automatic mode control module duringrun-time, so that during the run-time of the graphics-based application,the automatic mode control module can access and use the scene profilesmaintained in the application/scene profile database and determine howto dynamically control the modes of parallel operation of the MMPGRS tooptimize system performance.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering and display system, wherein the automaticmode control module employs real-time detection of scene profile indicesdirectly programmed within pre-profiled scenes of the graphics-basedapplication; wherein the pre-profiled scenes are analyzed prior torun-time, and indexed with the scene profile indices; and wherein andmode control parameters (MCPs) corresponding to the scene profileindices, are stored within an application/scene profile databaseaccessible to the automatic mode control module during applicationrun-time.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering and display system, wherein during run-time,the automatic mode control module automatically detects the sceneprofile indices and uses the detected said scene profile indices toaccess corresponding MCPs from the application/scene profile database soas to determine how to dynamically control the modes of paralleloperation of the MMPGRS to optimize system performance.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering and display system, wherein the automaticmode control module employs real-time detection of mode control commands(MCCs) directly programmed within pre-profiled scenes of thegraphics-based application; wherein the pre-profiled scenes are analyzedprior to run-time, and the MCCs are directly programmed within theindividual image frames of each scene; and wherein during run-time, theautomatic mode control module automatically detects the MCCs along thegraphics command and data stream, and uses the MCCs so as to determinehow to dynamically control the modes of parallel operation of the MMPGRSto optimize system performance.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering and display system, wherein the automaticmode control module employs a user interaction detection (UID) mechanismfor real-time detection of the user's interaction with the hostcomputing system.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering and display system, wherein, in conjunctionwith scene profiling, the automatic mode control module also uses saidUID mechanism to determine how to dynamically control the modes ofparallel operation of the MMPGRS to optimize system performance, at anyinstance in time during run-time of the graphics-based application.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system (MMPGRS), having multiple graphicsprocessing pipelines (GPPLs) with multiple GPUs supporting a parallelgraphics rendering process having time, frame and object division modesof operation, wherein each GPPL comprises video memory and a GPU havinga geometry processing subsystem and a pixel processing subsystem, andwherein 3D scene profiling is performed in real-time, and theparallelization state/mode of the system is dynamically controlled tomeet graphics application requirements.

Another object of the present invention is to provide a multi-modeparallel graphics rendering and display system having multiple graphicsprocessing pipelines (GPPLs), each having a GPU and video memory, andsupporting multiple modes of parallel graphics rendering using real-timegraphics application profiling and automatic configuration of themultiple graphics processing pipelines supporting multiple modes ofparallel graphics rendering, including a time-division mode, aframe-division mode, and an object-division mode of parallel operation.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering and display system, which is capable ofdynamically handling bottlenecks that are automatically detected duringany particular graphics application running on the host computingsystem.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering system, wherein different parallelizationschemes are employed to reduce pipeline bottlenecks, and increasegraphics performance.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering system, wherein image, time and objectdivision methods of parallelization are implemented on the same parallelgraphics platform.

Another object of the present invention is to provide a method ofmulti-mode parallel graphics rendering that can be practiced on amultiple GPU-based PC-level graphics system, and which, duringapplication run-time, dynamically alternates among Time, Frame/image andObject division modes of parallel operation, adapting the optimal methodof parallel operation to the real time needs of the graphicsapplication.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering system, which is capable of supervising theperformance level of a graphic application by dynamically adaptingdifferent parallelization schemes to solve instantaneous bottlenecksalong the graphic pipelines thereof.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering system, having run-time configurationflexibility for various parallel schemes to achieve the best systemperformance.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering system having architectural flexibility andreal-time profiling and control capabilities which enable utilization ofdifferent modes of parallel operation for high and steady performancealong the application running on the associated host system.

Another object of the present invention is to provide a novel method ofmulti-mode parallel graphics rendering on a multiple GPU-based graphicssystem, which achieves improved system performance by using adaptiveparallelization of multiple graphics processing units (GPUs), onconventional and non-conventional platform architectures, as well as onmonolithic platforms, such as multiple GPU chips or integrated graphicdevices (IGD).

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, wherein bottlenecks are dynamicallyhandled.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering system, wherein stable performance ismaintained throughout course of a graphics application.

Another object of the present invention to provide a multi-mode parallelgraphics rendering system supporting software-based adaptive graphicsparallelism for the best performance, seamlessly to the graphicsapplication, and compliant with graphic standards (e.g. OpenGL andDirect3D).

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, wherein all parallel modes areimplemented in a single architecture.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, wherein the architecture isflexible, supporting fast inter-mode transitions.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system which is adaptive to changing to meetthe needs of any graphics application during the course of itsoperation.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system which employs a user interactiondetection (UID) subsystem for enabling the automatic and dynamicdetection of the user's interaction with the host computing system.

Another object of the present invention is to provide such a multi-modeparallel graphics rendering system, continuously processes user-systeminteraction data, and automatically detects user-system interactivity(e.g. mouse click, keyboard depression, eye-movement, etc).

Another object of the present invention is to provide such a multi-modeparallel graphics rendering system the system, wherein absent preventiveconditions (such as CPU bottlenecks and need for the same FB insuccessive frames), the user interaction detection (UID) subsystemenables timely implementation of the Time Division Mode only when nouser-system interactivity is detected so that system performance isautomatically optimized.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, which can be implemented using asoftware implementation of present invention.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, which can be realized using ahardware implementation.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, can be realized as chipimplementation.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, which can be realized as anintegrated monolithic implementation.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, which can be implemented using IGDtechnology.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, characterized by run-timeconfiguration flexibility for various parallel schemes to achieve thebest parallel performance.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system that operates seamlessly to theapplication and is compliant with graphic standards (e.g. OpenGL andDirect3D).

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, which can be implemented onconventional multi-GPU platforms replacing image division or timedivision parallelism.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, which enables the multiple GPUplatform vendors to incorporate the solution in their systems supportingonly image division and time division modes of operation.

Another object of the present invention is to provide such multipleGPU-based graphics system, which enables implementation using low costmulti-GPU cards.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system implemented using IGD technology, andwherein it is impossible for the IGD to get disconnected by the BIOSwhen an external graphics card is connected and operating.

Another object of the present invention is to provide a multipleGPU-based graphics system, wherein a new method of dynamicallycontrolled parallelism improves the system's efficiency and performance.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, which can be implemented using anIGD supporting more than one external GPU.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, which can be implemented using anIGD-based chipset having two or more IGDs.

Another object of the present invention is to provide a multi-modeparallel graphics rendering system, which employs a user interactiondetection (UID) subsystem that enables automatic and dynamic detectionof the user's interaction with the system, so that absent preventiveconditions (such as CPU bottlenecks and need for the same FB insuccessive frames), this subsystem enables timely implementation of theTime Division Mode only when no user-system interactivity is detected,thereby achieving the highest performance mode of parallel graphicsrendering at runtime, and automatically optimizing the graphicsperformance of the host computing system.

Another object of the present invention is to provide a parallelgraphics rendering system employing multiple graphics processingpipelines supporting the object division mode of parallel graphicsrendering using pixel processing resources provided therewithin.

Another object of the present invention is to provide a parallelgraphics rendering system for carrying out the object division method ofparallel graphics rendering on multiple GPU-based graphics platformsassociated with diverse types of computing machinery.

Another object of the present invention is to provide a novel methodhaving multiple graphics processing pipelines (GPPLs) with multiple GPUsor CPU-cores supporting a parallel graphics rendering process having anobject division mode of operation, wherein each GPPL includes videomemory, a geometry processing subsystem, and a pixel processingsubsystem, wherein pixel (color and z depth) data buffered in the videomemory of each GPPL is communicated to the video memory of a primaryGPPL, and wherein the video memory and the pixel processing subsystem inthe primary GPPL are used to carry out the image recomposition phase ofthe object division mode of parallel graphics rendering process.

Another object of the present invention is to provide a parallelgraphics rendering system having multiple graphics processing pipelines(GPPLs) with multiple GPUs or CPU-cores supporting a parallel graphicsrendering process having an object division mode of operation, whereineach GPU comprises video memory, a geometry processing subsystem and apixel processing subsystem, wherein pixel (color and z depth) databuffered in the video memory of each GPPL is communicated to the videomemory of a primary GPPL, and wherein the video memory and the pixelprocessing subsystem in the primary GPPL are used to carry out the imagerecomposition phase of the object division mode of the parallel graphicsrendering process.

Another object of the present invention is to provide a parallelgraphics rendering system having multiple graphics processing pipelines(GPPLs) with multiple GPUs supporting a parallel graphics renderingprocess having an object division mode of operation, wherein each GPUcomprises video memory, a geometry processing subsystem and a pixelprocessing subsystem, wherein pixel (color and z depth) data buffered inthe video memory of each GPU is communicated to the video memory of aprimary GPU, and wherein the video memory and both the geometry andpixel processing subsystems in the primary GPU are used to carry out theimage recomposition phase of the object division mode of parallelgraphics rendering process.

Another object of the present invention is to provide a parallelrendering graphics system having multiple graphics processing pipelines(GPPLs) with multiple GPUs supporting a parallel graphics renderingprocess having an object division mode of operation, wherein the videomemory of each GPPL includes texture memory and a pixel frame buffer,wherein the geometry processing subsystem includes a vertex shadingunit, wherein the pixel processing subsystem includes a fragment/pixelshading unit, wherein pixel (color and z depth) data buffered in thevideo memory of each GPPL is communicated to the video memory of aprimary GPPL, and wherein the texture memory and the fragment/pixelshading unit are used to carry out the image recomposition phase of theobject division mode of the parallel graphics rendering process.

Another object of the present invention is to provide a parallelgraphics rendering system having multiple graphics processing pipelines(GPPLs) with multiple GPUs supporting a parallel graphics renderingprocess having an object division mode of operation, wherein the videomemory of each GPPL includes texture memory and a pixel frame buffer,wherein the geometry processing subsystem includes a vertex shadingunit, wherein the pixel processing subsystem includes a fragment/pixelshading unit, wherein pixel (color and z depth) data buffered in thevideo memory of each GPPL is communicated to the video memory of aprimary GPPL, and wherein the texture memory and the vertex shading unitare used to carry out the image recomposition phase of the objectdivision mode of the parallel graphics rendering process.

Another object of the present invention is to provide a parallelgraphics rendering system having multiple graphics processing pipelines(GPPLs) with multiple GPUs supporting a parallel graphics renderingprocess having an object division mode of operation, which does notrequire compositing in main, shared or distributed memory of the hostcomputing system (e.g. involving the movement of pixel data from theframe buffers or FBs to main memory, processing the pixel data in theCPU of the host for composition, and moving the result out to theprimary GPPL for display) thereby avoiding the use of expensiveprocedure and resources of the system (e.g. buses, caches, memory, andCPU bandwidth).

Another object of the present invention is to provide a novel method ofoperating a parallel graphics rendering system having multiple graphicsprocessing pipelines (GPPLs) with multiple GPUs supporting a parallelgraphics rendering process having an object division mode of operation,wherein implementation of the pixel composition phase of the parallelgraphics rendering process is carried out using the computationalresources within the GPUs, thereby avoiding the need for dedicated orspecialized pixel image compositing hardware and/or software basedapparatus.

Another object of the present invention is to provide a novel method ofobject division parallel graphics rendering carried out on a multi-modeparallel graphics rendering system (MMPGRS) or platform supportingmultiple graphical processing pipelines (GPPLs) with multiple graphicalprocessing units (GPUs), wherein the recomposition stage of therendering process is carried out using computational resources (e.g.video memory and the geometry and/or pixel processingsubsystems/engines) supplied by the GPPLs employed on the MMPGRSplatform.

Another object of the present invention is to provide a novel method ofobject division parallel rendering of pixel-composited images forgraphics-based applications running on a host computing system embodyinga multi-mode parallel graphics rendering system or platform (MMPGRS),wherein the movement and merging of composited pixel data occurs duringthe recomposition stage of the parallel graphics rendering process in amanner that is transparent to the graphics-based application.

Another object of the present invention is to provide a novel parallelrendering graphics system having multiple graphics processing pipelines(GPPLs) supporting a parallel graphics rendering process having anobject division mode of operation, wherein each GPPL comprises videomemory, a geometry processing subsystem and a pixel processingsubsystem, wherein pixel (color and z depth) data buffered in the videomemory of each GPPL is communicated (via an inter-GPPL communicationprocess) to the video memory of a primary GPPL, and wherein the videomemory and the geometry and/or pixel processing subsystems in theprimary GPPL are used to carry out the image recomposition phase of theobject division mode of the parallel graphics rendering process.

Another object of the present invention is to provide a novel parallelgraphics rendering system supporting multiple modes of paralleloperation during graphical rendering which allows users to enjoy sharpvideos and photos, smooth video playback, astonishing effects, andvibrant colors, as well as texture-rich 3D performance innext-generation games.

Another object of the present invention is to provide a novel multi-usercomputer network supporting a plurality of client machines, wherein eachclient machine employs the MMPGRS of the present invention based on asoftware architecture and responds to user-interaction input datastreams from one or more network users who might be local to each otheras over a LAN, or be remote to each other, as when operating over a WANor the Internet infrastructure.

Another object of the present invention is to provide a novel multi-usercomputer network supporting a plurality of client machines, wherein eachclient machine employs the MMPGRS of the present invention based on ahardware architecture and responds to user-interaction input datastreams from one or more network users who might be local to each otheras over a LAN, or be remote to each other, as when operating over a WANor the Internet infrastructure.

Another object of the present invention is to provide an Internet-basedcentral application profile database server system for automaticallyupdating, over the Internet, graphic application profiles (GAPs) withinthe MMPGRS of client machines.

Another object of the present invention is to provide suchInternet-based Central Application Profile Database Server System whichensures that each MMPGRS is optimally programmed at all possible timesso that it quickly and continuously offers users high graphicsperformance through its adaptive multi-modal parallel graphicsoperation.

Another object of the present invention is to provide such anInternet-based Central Application Profile Database Server System whichsupports a Web-based Game Application Registration and ProfileManagement Application, that provides a number of Web-based services,including:

(1) the registration of Game Application Developers within the RDBMS ofthe Server System;

(2) the registration of game applications with the RDBMS of the CentralApplication Profile Database Server System, by registered gameapplication developers;

(3) the registration of each MMPGRS deployed on a client machine orserver system having Internet-connectivity, and requesting subscriptionto periodic/automatic Graphic Application Profile (GAP) Updates(downloaded to the MMPGRS over the Internet) from the CentralApplication Profile Database Server System; and

(4) the registration of each deployed MMPGRS requesting the periodicuploading of its Game Application Profiles (GAPS)—stored in anApplication/Scene Profile Database and Historical Repository—to theCentral Application Profile Database Server System for the purpose ofautomated analysis and processing so as to formulate “expert” GameApplication Profiles (GAPs) that have been based on robustuser-experience and which are optimized for particular client machineconfigurations.

Another object of the present invention is to provide such anInternet-based Central Application Profile Database Server System thatenables the MMGPRS of registered client computing machines toautomatically and periodically upload, over the Internet, GraphicApplication Profiles (GAPs) for storage and use within theApplication/Scene Profile Database of the MMPGRS.

Another object of the present invention is to provide such anInternet-based Central Application Profile Database Server System which,by enabling the automatic uploading of expert GAPs into the MMPGRS,graphic application users (e.g. gamers) can immediately enjoy highperformance graphics on the display devices of their client machines,without having to develop a robust behavioral profile based on manyhours of actual user-system interaction.

Another object of the present invention is to provide such anInternet-based Central Application Profile Database Server System,wherein “expert” GAPs are automatically generated by the CentralApplication Profile Database Server System by analyzing the GAPs ofthousands of different game application users connected to the Internet,and participating in the system.

Another object of the present invention is to provide such anInternet-based Central Application Profile Database Server System,wherein for MMPGRS users subscribing to the Automatic GAP ManagementServices, each such MMPGRS runs an application profiling and controlalgorithm that uses the most recently uploaded expert GAP loaded intoits automatic mode control mechanism (AMCM), and then allow system-userinteraction, user behavior, and application performance to modify theexpert GAP profile over time until the next update occurs.

Another object of the present invention is to provide such anInternet-based Central Application Profile Database Server System,wherein the Application Profiling and Analysis Module in each MMGPRSsubscribing to the Automatic GAP Management Services supported by theCentral Application Profile Database Server System of the presentinvention, modifies and improves the downloaded expert GAP withinparticularly set limits and constraints, and according to particularcriteria, so that the expert GAP is allowed to evolve in an optimalmanner, without performance regression.

These and other objects of the present invention will become apparenthereinafter and in the claims to invention.

BRIEF DESCRIPTION OF DRAWINGS OF PRESENT INVENTION

For a more complete understanding of how to practice the Objects of thePresent Invention, the following Detailed Description of theIllustrative Embodiments can be read in conjunction with theaccompanying Drawings, briefly described below:

FIG. 1A 1 is a graphical representation of a typical prior art PC-basedcomputing system employing a conventional graphics architecture drivinga single external graphic card 105;

FIG. 1A 2 a graphical representation of a conventional GPU subsystemsupported on the graphics card of the PC-based graphics system of FIG.1A 1;

FIG. 1A 3 is a graphical representation illustrating the generalsoftware architecture of the prior art computing system shown in FIG. 1A2;

FIG. 1B 1 is a graphical representation of a typical prior art PC-basedcomputing system employing a conventional graphics architectureemploying a North memory bridge circuit (i.e. semiconductor chip ofmonolithic construction) with an integrated graphics device (IGD) 103supporting a single graphics pipeline process, and being operablycoupled to a South bridge circuit (i.e. semiconductor chip of monolithicconstruction) supporting the input/output ports of the system;

FIG. 1B 2 is graphical representation of the North memory bridgeemployed in the system of FIG. 1B 1, showing in greater detail themicro-architecture of the IGD supporting the single graphics pipelineprocess therewithin;

FIG. 1B 3 is a graphical representation illustrating the generalsoftware architecture of the prior art PC-based IGD-driven computingsystem shown in FIGS. 1B1 and 1B2;

FIG. 1B 4 is a graphical representation of a prior art PC-basedcomputing system employing a conventional Fusion-type CPU/GPU hybridarchitecture, wherein a single GPU 1242 implemented on the samesemiconductor die as the CPU 1241 is used to support a graphics pipelinethat drives an external display device, e.g. LCD panel, projectiondisplay or the like 106, via a bridge circuit, with display interface,as shown;

FIG. 1B 5 is a schematic representation showing the structure of a priorart GPU subsystem mounted on a graphics card or in an IGD, andcomprising a GPU, and a video memory which is external to the GPU,wherein the GPU has includes two 3D engines, namely, (i) a transformbound geometry subsystem 124 for processing 3D graphics primitives 121,and (ii) and a fill bound pixel subsystem 125, and wherein the videomemory shares its storage resources among a geometry buffer 122A throughwhich all geometric (i.e. polygonal) data 121 is transferred, a commandsbuffer 122B, texture buffers 123, and a Frame Buffer 126;

FIG. 1C is a graphical representation illustrating a conventionalprocess for rendering successive 3D scenes using a single GPU graphicsplatform to support a single graphics pipeline process, as shown inFIGS. 1A1 through 1B5;

FIG. 2A 1 is a graphical representation of a prior art PC-basedcomputing system employing a conventional dual-GPU graphic architecturecomprising two external graphic cards 205 204 and two PCI-e buses, e.g.Bearlake by Intel 207, 208, wherein the primary and secondary graphicscards are connected to and driven by the North memory bridge circuit103, while a display device 106 is attached to the primary graphics card205, and Ethernet and mobile docking ports and other local I/O ports aredriven by the South bridge circuit, as shown;

FIG. 2A 2 is a graphical representation of a prior art PC-basedcomputing system employing a conventional dual-GPU graphic architecturecomprising two external graphic cards 204, 205 and two PCI-e buses (e.g.Bearlake by Intel), wherein the primary graphics card 205 is connectedto and driven by the North memory bridge via the first PCI-e bus with adisplay device 106 is attached to the primary graphics card 205, andwherein the secondary graphics card 204 is connected to and driven bythe South bridge via the second PCI-e bus;

FIG. 2A 3 is a graphical representation of a conventional GPU subsystemsupported on each of the graphics cards employed in the prior artPC-based computing systems of FIGS. 2A1 and 2A2;

FIG. 2A 4 is a graphical representation illustrating the generalsoftware architecture of the prior art PC-based graphics systems shownin FIG. 2A 1, as well as FIG. 2A 2;

FIG. 2A 5 is a graphical representation of a prior art PC-basedcomputing system employing a conventional multi-core microprocessor(CPU) chip to implement multiple processing cores in a single physicalpackage, wherein some of the cores can be potentially used as softgraphic graphics pipelines, and wherein a display device 106 isconnected to and driven by the North (memory) bridge chip on themotherboard;

FIG. 2B is a graphical representation of a conventional parallelgraphics rendering process being carried out according to the ImageDivision Method of parallelism using the dual GPUs provided on the priorart graphics platform illustrated in FIGS. 2A1 through 2A3;

FIG. 2C is a graphical representation of a conventional parallelgraphics rendering process being carried out according to the TimeDivision Method of parallelism using the dual GPUs provided on the priorart graphics platforms illustrated in FIGS. 2A1 through 2A5;

FIG. 3A is a schematic representation of a prior art parallel graphicsrendering platform comprising multiple parallel graphics pipelines, eachsupporting video memory and a GPU, and feeding complex pixel compositinghardware for composing a final pixel-based images for display on thedisplay device;

FIG. 3B is a graphical representation of a conventional parallelgraphics rendering process being carried out according to the ObjectDivision Method of parallelism using multiple GPUs on the prior artgraphics platform of FIG. 3A;

FIG. 3C 1 is a schematic representation of the GPU and Video Memorystructure employed in conventional multi-GPU PC-based computing systems,and illustrating the various kinds of bottlenecks (e.g. geometrylimited, pixel limited, data transfer limited, and memory limited) thatoccur in such systems;

FIG. 3C 2 is a table summarizing the kinds of bottleneck problems whichconventional parallelization modes are currently capable of mitigatingalong the multi-GPU pipelines of conventional PC-based computingsystems;

FIG. 4A is a schematic representation of a generalized embodiment of themulti-mode parallel 3D graphics rendering system (MMPGRS) of the presentinvention shown comprising (i) an automatic mode control module ormechanism (AMCM) 400 for supporting automatic mode control using diversetypes of 3D scene profiling techniques and/or system-user interactiondetection techniques, (ii) a multi-mode parallel graphics renderingsubsystem 41 for supporting at least three primary parallelizationstages of decomposition, distribution and recomposition implementedusing the Decomposition Module 401, the Distribution Module 402 and theRecomposition Module 403, respectively, and (ii) a plurality of eitherGPU and/or CPU based “graphics processing pipelines (GPPLs)” 410′,wherein each parallelization stage performed by its corresponding Moduleis configured (i.e. set up) into a “sub-state” by set of parameters, andwherein the “graphics rendering parallelism state” for the overallmulti-mode parallel graphics system is established or otherwisedetermined by the combination of sub-states of these component stages;

FIG. 4B 1 is a schematic representation of the subcomponents of a firstillustrative embodiment of a GPU-based graphics processing pipeline(GPPL) that can be employed in the MMPGRS of the present inventiondepicted in FIG. 4A, shown comprising (i) a video memory structuresupporting a frame buffer (FB) including stencil, depth and colorbuffers, and (ii) a graphics processing unit (GPU) supporting (1) ageometry subsystem having an input assembler and a vertex shader, (2) aset up engine, and (3) a pixel subsystem including a pixel shaderreceiving pixel data from the frame buffer and a raster operatorsoperating on pixel data in the frame buffers;

FIG. 4B 2 is a schematic representation of the subcomponents of a secondillustrative embodiment of a GPU-based graphics processing pipeline(GPPL) that can be employed in the MMPGRS of the present inventiondepicted in FIG. 4A, shown comprising (i) a video memory structuresupporting a frame buffer (FB) including stencil, depth and colorbuffers, and (ii) a graphics processing unit (GPU) supporting (1) ageometry subsystem having an input assembler, a vertex shader and ageometry shader, (2) a rasterizer, and (3) a pixel subsystem including apixel shader receiving pixel data from the frame buffer and a rasteroperators operating on pixel data in the frame buffers;

FIG. 4B 3 is a schematic representation of the subcomponents of aillustrative embodiment of a CPU-based graphics processing pipeline thatcan be employed in the MMPGRS of the present invention depicted in FIG.4A, shown comprising (i) a video memory structure supporting a framebuffer including stencil, depth and color buffers, and (ii) a graphicsprocessing pipeline realized by one cell of a multi-core CPU chip,consisting of 16 in-order SIMD processors, and further including aGPU-specific extension, namely, a texture sampler that loads texturemaps from memory, filters them for level-of-detail, and feeds to pixelprocessing portion of the pipeline;

FIG. 4C is a schematic representation for the Mode Definition Tablewhich shows the four combinations of sub-modes (i.e. sub-states) A:B:Cfor realizing the three (3) Parallel Modes of the MMPGRS of the presentinvention (i.e. Object Division Mode, Image Division Mode andTime/Alternative Division Mode), and the one (1) Single GPU (i.e.Non-Parallel Functioning) Mode of the system;

FIG. 4D is a schematic representation illustrating the variousPerformance and Interactive Device Data Inputs supplied to theApplication Profiling and Analysis Module (within the Automatic ModeControl Module (AMCM)) employed in the MMPGRS of present invention shownin FIG. 4A, as well as the Tasks carried out by the ApplicationProfiling and Analysis Module,

FIG. 5A is a schematic representation of the User Interaction Detection(UID) Subsystem employed within the Application Profiling and AnalysisModule of the Automatic Mode Control Module (AMCM) in the MMPGRS of thepresent invention, wherein the UID Subsystem is shown comprising aDetection and Counting Module arranged in combination with a UIDTransition Decision Module;

FIG. 5B is a flow chart representation of the state transition processbetween Object-Division/Image-Division Modes and the Time Division Modeinitiated by the UID subsystem employed in the MMPGRS of the presentinvention;

FIG. 5C 1 is a schematic representation of the process carried out bythe Profiling and Control Cycle in the Automatic Mode Control Module(AMCM) in the MMPGRS of present invention, while the UID Subsystem isdisabled;

FIG. 5C 2 is a schematic representation of the process carried out bythe Profiling and Control Cycle in the Automatic Mode Control Module inthe MMPGRS of present invention, while the UID Subsystem is enabled;

FIG. 5C 3 is a schematic representation of the process carried out bythe Periodical Trial & Error Based Control Cycle in the Automatic ModeControl Module employed in the MMPGRS of present invention, shown inFIG. 4A;

FIG. 5C 4 is a schematic representation of the process carried out bythe Event Driven Trial & Error Control Cycle in the Automatic ModeControl Module employed in the MMPGRS of present invention, shown inFIG. 4A;

FIG. 6A is a State Transition Diagram for the MMPGRS of presentinvention, illustrating that a parallel state is characterized by A, B,C sub-state parameters, that the non-parallel state (single GPPL) is anexceptional state, reachable from any state by a graphics application orAMCM requirement, and that all state transitions in the system arecontrolled by Automatic Mode Control Module (AMCM), wherein in thosecases of known and previously analyzed graphics applications, the AMCM,when triggered by events (e.g. drop in frames per second FPS rate),automatically consults the Application/Scene Profile Database during thecourse of the Application, or otherwise, makes decisions which aresupported by continuous profiling and analysis of listed parameters,and/or trial and error event driven or periodical cycles;

FIG. 6B is a schematic representation of the MMPGRS of the presentinvention supporting multiple graphic processing pipelines (GPPLs), withdynamic application profiling and parallelism mode control, inaccordance with the principles of the present invention;

FIG. 6C 1 is a flow chart illustrating the processing of a sequence ofpipelined image frames during the Image Division Mode of parallelgraphics rendering supported on the MMPGRS of the present inventiondepicted in FIGS. 4A through 6A;

FIG. 6C 2 is a flow chart illustrating the processing of a sequence ofpipelined image frames during the Time Division Mode of parallelgraphics rendering supported on the MMPGRS of the present inventiondepicted in FIGS. 4A through 6A;

FIG. 6C 3 is a flow chart illustrating the processing of a single imageframe during the Object Division mode of parallel graphics renderingsupported on the MMPGRS of the present invention depicted in FIGS. 4Athrough 6A;

FIG. 7A 1-1 is a schematic representation of various possible graphicsarchitectural spaces within which the components of the MMPGRS of thepresent invention can be embodied in any given application, namely: HostMemory Space (HMS), Processor/CPU Die Space, Bridge Circuit (IGD) Space,Graphics Hub Space, and External GPU Space;

FIG. 7A 1-2 sets forth a table listing diverse classes of systemarchitectures in which the MMPGRS can be embodied, expressed in terms ofthe different kinds of architectural spaces, identified in FIG. 7A 1-1,in which the primary MMPGRS components (i.e. AMCM, DecompositionSubmodule 1, Decomposition Module 2, Distribution Module, Multiple GPUsand Recomposition Module) can be embodied in each such class of MMPGRSArchitecture, namely—Host Memory Space HMS (software), HMS+IGD,HMS+Fusion, HMS+Multicore, HMS+GPU-Recomposition, HUB;HUB+GPU-Recomposition, Chipset; CPU/GPU Fusion, Multicore CPU, and GameConsole;

FIG. 7A 2 is a schematic representation of a first illustrativeembodiment of the MMPGRS of the present invention, following the HMSClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the Decomposition,Distribution and Recomposition Modules 401, 402, 403, respectively, ofthe Multimode Parallel Graphics Rendering Subsystem resides as asoftware package 701 in the Host Memory Space (HMS) while multiple GPUsare supported on a pair of external graphic cards 204, 205 connected toa North memory bridge chip 103 and driven in a parallelized manner underthe control of the AMCM, (ii) the Decomposition Module 401 divides (i.e.splits up) the stream of graphic commands and data (GCAD) according tothe required parallelization mode, operative at any instant in time,(iii) the Distribution Module 402 uses the North bridge chip todistribute graphic commands and data (GCAD) to the multiple GPUs onboard the external graphics cards, (iv) the Recomposition Module 403uses the North bridge chip to transfer composited pixel data (CPD)between the Recomposition Module (or CPU) and the multiple GPUs duringthe image recomposition stage, and (v) finally recomposited pixel datasets are displayed as graphical images on one or more display devicesconnected to the external graphics cards via a PCI-express interface(which is connected to the North bridge chip);

FIG. 7A 3 is a schematic representation of a second illustrativeembodiment of the MMPGRS of the present invention, following the HMS+IGDClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the Decomposition,Distribution and Recomposition Modules 401, 402, 403, respectively, ofthe Multimode Parallel Graphics Rendering Subsystem resides as asoftware package 701 in the Host or CPU Memory Space (HMS) whilemultiple GPUs are supported in an IGD within the North memory bridgecircuit as well as on external graphic cards connected to the Northmemory bridge chip and driven in a parallelized manner under the controlof the AMCM, (ii) the Decomposition Module 401 divides (i.e. splits up)the stream of graphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iii) theDistribution Module 402 uses the North bridge chip to distribute thegraphic commands and data (GCAD) to the multiple GPUs located in the IGDand on the external graphics cards, (iv) the Recomposition Module 403uses the North bridge chip to transfer composited pixel data (CPD)between the Recomposition Module (or CPU) and the multiple GPUs duringthe image recomposition stage, and (v) finally recomposited pixel datasets are displayed as graphical images on one or more display devicesconnected to one of the external graphics cards or the IGD;

FIG. 7A 4 is a schematic representation of a third illustrativeembodiment of the MMPGRS of the present invention, following the HMS+IGDClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the Decomposition,Distribution and Recomposition Modules 401, 402, 403, respectively, ofthe Multimode Parallel Graphics Rendering Subsystem resides as asoftware package 701 in the Host Memory Space (HMS) while multiple GPUsare supported in an IGD within the South bridge circuit as well as onexternal graphic cards connected to the South bridge chip, and driven ina parallelized manner under the control of the AMCM, (ii) theDecomposition Module 401 divides (i.e. splits up) the stream of graphiccommands and data (GCAD) according to the required parallelization mode,operative at any instant in time, (iii) that the Distribution Module 402uses the North bridge chip to distribute graphic commands and data(CGAD) to the multiple GPUs located in the IGD and on external graphicscards, (iv) the Recomposition Module 403 uses the South bridge chip totransfer recomposited pixel data between the Recomposition Module (orCPU) and the multiple GPUs during the image recomposition stage, and (v)finally recomposited pixel data sets are displayed as graphical imageson one or more display devices connected to one of the external graphicscards or the IGD;

FIG. 7A 5 is a schematic representation of a fourth illustrativeembodiment of the MMPGRS of the present invention, following theHMS+Fusion Class of MMPGRS Architecture described in FIG. 7A 1-2, andshowing (i) that the Automatic Mode Control Module (AMCM) 400 and theDecomposition, Distribution and Recomposition Modules 401, 402, 403,respectively, of the Multimode Parallel Graphics Rendering Subsystemresides as a software package 701 in the Host Memory Space (HMS) while asingle GPU 1242 is supported on a CPU/GPU fusion-architecture processordie (alongside the CPU 1241) and one or more GPUs are supported on anexternal graphic card connected to the CPU processor die and driven in aparallelized manner under the control of the AMCM, (ii) theDecomposition Module 401 divides (i.e. splits up) the stream of graphiccommands and data (GCAD) according to the required parallelization mode,operative at any instant in time, (iii) the Distribution Module 402 usesthe memory controller (controlling the HMS) and the interconnect network(e.g. crossbar switch) within the CPU/GPU processor chip to distributegraphic commands and data to the multiple GPUs on the CPU/GPU die chipand on the external graphics cards, (iv) the Recomposition Module 403uses the memory controller and interconnect (e.g. crossbar switch)within the CPU/GPU processor chip to transfer composited pixel data(CPD) between the Recomposition Module (or CPU) and the multiple GPUsduring the image recomposition stage, and (v) finally recomposited pixeldata sets are displayed as graphical images on one or more displaydevices connected to the external graphics card via a PCI-expressinterface (which is connected to the CPU/GPU fusion-architecture chip);

FIG. 7A 6 is a schematic representation of a fifth illustrativeembodiment of the MMPGRS of the present invention, following theHMS+Multicore Class of MMPGRS Architecture described in FIG. 7A 1-2, andshowing (i) that the Automatic Mode Control Module (AMCM) 400 and theDecomposition, Distribution and Recomposition Modules 401, 402, 403,respectively, of the Multimode Parallel Graphics Rendering Subsystemresides as a software package 701 in the Host or CPU Memory Space (HMS)while some of the CPU cores on a multi-core CPU chip are used toimplement a plurality of multi-core graphics pipelines parallelizedunder the control of the AMCM, (ii) the Decomposition Module 401 divides(i.e. splits up) the stream of graphic commands and data (GCAD)according to the required parallelization mode, operative at any instantin time, (iii) the Distribution Module 402 uses the North memory bridgeand interconnect network within the multi-core CPU chip to distributegraphic commands and data (GCAD) to the multi-core graphic pipelinesimplemented on the multi-core CPU chip, (iv) the Recomposition Module403 uses the North memory bridge and interconnect network within themulti-core CPU chip to transfer composited pixel data (CPD) between theRecomposition Module (or CPU) and the multi-core graphics pipelinesduring the image recomposition stage, and (v) finally recomposited pixeldata sets are displayed as graphical images on one or more displaydevices connected to the North bridge chip via a display interface;

FIG. 7A 7 is a schematic representation of a sixth illustrativeembodiment of the MMPGRS of the present invention, following theHMS+GPU-Recomposition Class of MMPGRS Architecture described in FIG. 7A1-2, and showing (i) that the Automatic Mode Control Module (AMCM) 400and the Decomposition, Distribution and Recomposition Modules 401, 402respectively, of the Multimode Parallel Graphics Rendering Subsystemresides as a software package 701 in the Host or CPU Memory Space (HMS)while multiple GPUs on external GPU cards are used to implement theRecomposition Module, and driven in parallelized manner under thecontrol of the AMCM, (ii) the Decomposition Module 401 divides (i.e.splits up) the stream of graphic commands and data (GCAD) according tothe required parallelization mode, operative at any instant in time,(iii) the Distribution Module 402 uses the North or South bridge circuitand interconnect network to distribute graphic commands and data (GCAD)to the external GPUs, (iv) the Recomposition Module uses the Northmemory bridge and associated system bus (e.g. PCI-express bus) totransfer composited pixel data (CPD) between the GPUs during the imagerecomposition stage, and (v) finally recomposited pixel data sets(recomposited within the vertex and/or fragment shaders of the primaryGPU) are displayed as graphical images on one or more display devices,connected to an external graphics card via a PCI-express interface(which is connected to either the North or South bridge circuits of thehost computing system);

FIG. 7A 7-1 is a schematic representation of the parallel graphicsrendering process supported within the MMPGRS of FIG. 7A 7 during itsobject division mode of parallel operation;

FIG. 7A 7-2 is a graphical representation of Shader code (expressed in agraphics programming language, e.g. Cg) that is used within the primaryGPPL of the MMPGRS of FIG. 7A 7, in order to carry out the pixelrecomposition stage of the object division mode/method of the parallelgraphics rendering process of the present invention, supported on thedual GPU-based parallel graphics system shown in FIG. 7A 7;

FIG. 7A 7-3 is a time-line representation of process of generating aframe of pixels for an image along a specified viewing direction, duringa particular parallel rendering cycle in the MMPGRS of FIG. 7A 7,wherein the pixel recomposition step of the parallel rendering processis shown reusing GPU-based computational resources during its idle time,without the need for specialized or dedicated compositional apparatusrequired by prior art parallel graphics systems supporting an objectdivision mode of parallel graphics rendering;

FIG. 7B 1 is a schematic representation of a seventh illustrativeembodiment of the MMPGRS of the present invention, following the HubClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host or CPUMemory Space (HMS) while the Decomposition Submodule No. 2 401″,Distribution Module 402″ and Recomposition Module 403″ are realizedwithin a single graphics hub device (e.g. chip) that is connected to theNorth memory bridge of the host computing system via a PCI-expressinterface and to a cluster of external GPUs 410″ via an interconnect,with the GPUs being driven in a parallelized manner under the control ofthe AMCM, (ii) the Decomposition Submodule No. 1 transfers graphiccommands and data (GCAD) to the Decomposition Submodule No. 2 via theNorth memory bridge circuit, (iii) the Decomposition Submodule No. 2divides (i.e. splits up) the stream of graphic commands and data (GCAD)according to the required parallelization mode, operative at any instantin time, (iv) the Distribution Module 402″ distributes graphic commandsand data (GCAD) to the external GPUs, (v) the Recomposition Module 403″transfers composited pixel data (CPD) between the GPUs during the imagerecomposition stage, and (vi) finally recomposited pixel data sets aredisplayed as graphical images on one or more display devices connectedto the primary GPU on the graphical display card which is connected tothe graphics hub chip of the present invention via the interconnect404″;

FIG. 7B 2 is a schematic representation of an eighth illustrativeembodiment of the MMPGRS of the present invention, following theHub+GPU-Recomposition Class of MMPGRS Architecture described in FIG. 7A1-2, and showing (i) that the Automatic Mode Control Module (AMCM) 400and the Decomposition Submodule No. 1 401′ reside as a software packagein the Host Memory Space (HMS) of the host computing system, while theDecomposition Submodule No. 2 401″ and the Distribution Module 402″ arerealized within a single graphics hub device (e.g. chip) that isconnected to the North bridge of the host computing system and a clusterof external GPUs 410″, and the Recomposition Module (403″ is implementedacross two or more GPUs 715, 716 of the system, as taught in FIG. 7A 7),and that all of the GPUs are driven in a parallelized manner under thecontrol of the AMCM, (ii) the Decomposition Submodule No. 1 transfersgraphic commands and data (GCAD) to the Decomposition Submodule No. 2via the North bridge circuit, (iii) the Decomposition Submodule No. 2divides (i.e. splits up) the stream of graphic commands and data (GCAD)according to the required parallelization mode, operative at any instantin time, (iv) the Distribution Module 402″ distributes graphic commandsand data (GCAD) to the external GPUs, (v) the Recomposition Module 403″transfers composited pixel data (CPD) between the GPUs during the imagerecomposition stage, and (vi) finally recomposited pixel data sets(recomposited within the vertex and/or fragment shaders of the primaryGPU) are displayed as graphical images on one or more display devicesconnected to the primary GPU on the graphical display card(s) (which areconnected to the graphics hub chip of the present invention);

FIG. 7B 3 is a schematic representation of a ninth illustrativeembodiment of the MMPGRS of the present invention, following the ChipsetClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host or CPUMemory Space (HMS) while the Decomposition Submodule No. 2 401″,Distribution Module 402″ and Recomposition Module 403″ are realized (asa graphics hub) in an integrated graphics device (IGD) within the Northmemory bridge circuit and having a plurality of GPUs being driven in aparallelized manner under the control of the AMCM, (ii) theDecomposition Submodule No. 1 transfers graphic commands and data (GCAD)to the Decomposition Submodule No. 2 via the North bridge circuit, (iii)the Decomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iv) theDistribution Module 402″ distributes graphic commands and data (GCAD) tothe internal GPUs via the interconnect network, (v) the RecompositionModule 403″ transfers composited pixel data (CPD) between the GPUsduring the image recomposition stage, and (vi) finally recompositedpixel data sets are displayed as graphical images on one or more displaydevices connected to the external graphical display card or the primaryGPU in the IGB;

FIG. 7B 4 is a schematic representation of a tenth illustrativeembodiment of the MMPGRS of the present invention, following the ChipsetClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host or CPUMemory Space (FIMS) while the Decomposition Submodule No. 2 401″,Distribution Module 402″ and Recomposition Module 403″ are realized (asa graphics hub) in an integrated graphics device (IGD) within the Southbridge circuit of the host computing system and having a plurality ofGPUs driven in a parallelized manner under the control of the AMCM, (ii)the Decomposition Submodule No. 1 transfers graphic commands and data(GCAD) to the Decomposition Submodule No. 2 via the communicationinterfaces of the North and South bridge circuits, (iii) theDecomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iv) theDistribution Module 402″ distributes graphic commands and data (GCAD) tothe external GPUs, (v) the Recomposition Module 403″ transferscomposited pixel data (CPD) between the GPUs during the imagerecomposition stage, and (vi) finally recomposited pixel data sets aredisplayed as graphical images on one or more display devices connectedto the external graphical display card or the primary GPU in the IGB;

FIG. 7B 4-1 is a schematic representation of an eleventh illustrativeembodiment of the MMPGRS of the present invention, following the ChipsetClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host or CPUMemory Space (HMS) while the Decomposition Submodule No. 2 401″ andDistribution Module 402″ are realized (as a graphics hub) in anintegrated graphics device (IGD) within the South bridge circuit of thehost computing system and having a plurality of GPUs driven in aparallelized manner under the control of the AMCM, while theRecomposition Module 403″ is implemented across two or more GPUs 715,716, (ii) the Decomposition Submodule No. 1 transfers graphic commandsand data (GCAD) to the Decomposition Submodule No. 2 via thecommunication interfaces of the North and South bridge circuits, (iii)the Decomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iv) theDistribution Module 402″ distributes graphic commands and data (GCAD) tothe external GPUs, (v) the Recomposition Module 403″, implemented withinPrimary GPU, transfers composited pixel data (CPD) between the GPUsduring the image recomposition stage, and (vi) finally recompositedpixel data sets are displayed as graphical images on one or more displaydevices connected to the external graphical display card or the primaryGPU in the IGB;

FIG. 7B 5 is a schematic representation of an twelfth illustrativeembodiment of the MMPGRS of the present invention, following the ChipsetClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host or CPUMemory Space (HMS) while the Decomposition Submodule No. 2 401″,Distribution Module 402″ and Recomposition Module 403″ are realized (asa graphics hub) in an integrated graphics device (IGD) within the Northmemory bridge of the host computing system and having multiple GPUsbeing driven with a single GPU on an external graphics card in aparallelized manner under the control of the AMCM, (ii) theDecomposition Submodule No. 1 transfers graphics commands and data(GCAD) to the Decomposition Submodule No. 2 via the North bridgecircuit, (iii) the Decomposition Submodule No. 2 divides (i.e. splitsup) the stream of graphic commands and data (GCAD) according to therequired parallelization mode, operative at any instant in time, (iv)the Distribution Module 402″ distributes graphic commands and data(GCAD) to the external GPUs, (v) the Recomposition Module 403″ transferscomposited pixel data (CPD) between the GPUs during the imagerecomposition stage, and (vi) finally recomposited pixel data sets aredisplayed as graphical images on one or more display devices connectedto the external graphical display card or the primary GPU in the IGB;

FIG. 7B 6 is a schematic representation of a thirteenth illustrativeembodiment of the MMPGRS of the present invention, following the ChipsetClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host or CPUMemory Space (HMS) while the Decomposition Submodule No. 2 401″,Distribution Module 402″ and Recomposition Module 403″ are realized (asa graphics hub) in an integrated graphics device (IGD) within the Southbridge circuit of the host computing system and having a single GPUdriven with a single GPU on an external graphics card in a parallelizedmanner under the control of the AMCM, (ii) the Decomposition SubmoduleNo. 1 transfer graphic commands and data (GCAD) to the DecompositionSubmodule No. 2 via the North and South bridge circuits, (iii) theDecomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iv) theDistribution Module 402″ distributes the graphic commands and data(GCAD) to the external GPUs, (v) the Recomposition Module 403″ transferscomposited pixel data (CPD) between the GPUs during the imagerecomposition stage, and (vi) finally recomposited pixel data sets aredisplayed as graphical images on one or more display devices connectedto the external graphics card or the primary GPU in the IGB;

FIG. 7B 6-1 is a schematic representation of a fourteenth illustrativeembodiment of the MMPGRS of the present invention, following the ChipsetClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host or CPUMemory Space (HMS) while the Decomposition Submodule No. 2 401″ andDistribution Module 402″ are realized (as a graphics hub) in anintegrated graphics device (IGD) within the South bridge circuit of thehost computing system and having multiple GPUs driven with a single GPUon an external graphics card in a parallelized manner under the controlof the AMCM, while the Recomposition Module 403″ is implemented acrosstwo or more GPUs 715, 716, (ii) the Decomposition Submodule No. 1transfer graphic commands and data (GCAD) to the Decomposition SubmoduleNo. 2 via the North and South bridge circuits, (iii) the DecompositionSubmodule No. 2 divides (i.e. splits up) the stream of graphic commandsand data (GCAD) according to the required parallelization mode,operative at any instant in time, (iv) the Distribution Module 402″distributes the graphic commands and data (GCAD) to the external GPUs,(v) the Recomposition Module 403″ transfers composited pixel data (CPD)between the GPUs during the image recomposition stage, and (vi) finallyrecomposited pixel data sets are displayed as graphical images on one ormore display devices connected to one of the external graphics card orthe primary GPU in the IGB;

FIG. 7B 7 is a schematic representation of a fifteenth illustrativeembodiment of the MMPGRS of the present invention, following the ChipsetClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host or CPUMemory Space (HMS) while the Decomposition Submodule No. 2 401″,Distribution Module 402″ and Recomposition Module 403″ are realized (asa graphics hub) in an integrated graphics device (IGD) within the Northmemory bridge of the host computing system and having a single GPU beingdriven with one or more GPUs on multiple external graphics cards in aparallelized manner under the control of the AMCM (or alternatively (ii)controlling a single GPU aboard the IGD for driving a display deviceconnected to the IGD via a display interface), (ii) the DecompositionSubmodule No. 1 transfers graphic commands and data (GCAD) to theDecomposition Submodule No. 2 via the North bridge circuit, (iii) theDecomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iv) theDistribution Module 402″ distributes the graphic commands and data(GCAD) to the internal GPU and external GPUs, (v) the RecompositionModule 403″ transfers composited pixel data (CPD) between the GPUsduring the image recomposition stage, and (vi) finally recompositedpixel data sets are displayed as graphical images on one or more displaydevices connected to one of the external graphic cards or the primaryGPU in the IGB;

FIG. 7B 7-1 is a schematic representation of a sixteenth illustrativeembodiment of the MMPGRS of the present invention, following the ChipsetClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host or CPUMemory Space (HMS) while the Decomposition Submodule No. 2 401″ andDistribution Module 402″ are realized (as a graphics hub) in anintegrated graphics device (IGD) realized within the North memory bridgechip of the host computing system, and driving (a) multiple GPUs onmultiple external graphics cards in a parallelized manner under thecontrol of the AMCM while the Recomposition Module 403″ is implementedacross two or more GPUs 715, 716, or alternatively (b) controlling asingle GPU aboard the IGD for driving a display device connected to theIGD via a display interface, (ii) the Decomposition Submodule No. 1transfers graphic commands and data (GCAD) to the DecompositionSubmodule No. 2 via the North bridge circuit, (iii) the DecompositionSubmodule No. 2 divides (i.e. splits up) the stream of graphic commandsand data (GCAD) according to the required parallelization mode,operative at any instant in time, (iv) the Distribution Module 402″distributes the graphic commands and data (GCAD) to the internal GPU andexternal GPUs, (v) the Recomposition Module 403″ implemented in theprimary GPU, transfers composited pixel data (CPD) between the GPUsduring the image recomposition stage, and (vi) finally recompositedpixel data sets are displayed as graphical images on one or more displaydevices connected to one of the external graphic cards or the primaryGPU in the IGB;

FIG. 7B 8-1 is a schematic representation of a seventeenth illustrativeembodiment of the MMPGRS of the present invention, following theCPU/GPU_Fusion Class of MMPGRS Architecture described in FIG. 7A 1-2,and showing (i) that the Automatic Mode Control Module (AMCM) 400 andthe Decomposition Submodule No. 1 401′ reside as a software package inthe Host Memory Space (HMS) while the Decomposition Submodule No. 2401″, Distribution Module 402″ and Recomposition Module 403″ arerealized (as a graphics hub) on the die of a hybrid CPU/GPUfusion-architecture chip within the host computing system and having asingle GPU driven with one or more GPUs on an external graphics card(connected to the CPU/GPU chip) in a parallelized under the control ofthe AMCM, (ii) the Decomposition Submodule No. 1 transfers graphicscommands and data (GCAD) to the Decomposition Submodule No. 2, (iii) theDecomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iv) theDistribution Module 402″ distributes the graphic commands and data(GCAD) to the internal GPU and external GPUs, (v) the RecompositionModule 403″ transfers composited pixel data (CPD) between the GPUsduring the image recomposition stage, and (vi) finally recompositedpixel data sets are displayed as graphical images on one or more displaydevices 106 connected to the external graphics card connected to thehybrid CPU/GPU chip via a PCI-express interface;

FIG. 7B 8-2 is a schematic representation of an eighteenth illustrativeembodiment of the MMPGRS of the present invention, following the CPU/GPUClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host MemorySpace (HMS) while the Decomposition Submodule No. 2 401″, DistributionModule 402″ and Recomposition Module 403″ are realized (as a graphicshub) on the die of a hybrid CPU/GPU fusion-architecture chip within thehost computing system and having multiple GPUs 1242″ driven with one ormore GPUs on an external graphics card 205 (connected to the CPU/GPUchip) in a parallelized manner under the control of the AMCM, (ii) theDecomposition Submodule No. 1 transfers graphics commands and data(GCAD) to the Decomposition Submodule No. 2, (iii) the DecompositionSubmodule No. 2 divides (i.e. splits up) the stream of graphic commandsand data (GCAD) according to the required parallelization mode,operative at any instant in time, (iv) the Distribution Module 402″ usesthe crossbar switch (i.e. interconnect) on the processor die todistribute the graphic commands and data (GCAD) to the internal GPUs andexternal GPUs, (v) the Recomposition Module 403″ transfers compositedpixel data (CPD) between the GPUs during the image recomposition stage,and (vi) finally recomposited pixel data sets are displayed as graphicalimages on one or more display devices 106 connected to the externalgraphics card connected to the hybrid CPU/GPU chip via a PCI-expressinterface;

FIG. 7B 8-3 is a schematic representation of a nineteenth illustrativeembodiment of the MMPGRS of the present invention, following the CPU/GPUClass of MMPGRS Architecture described in FIG. 7A 1-2, and showing (i)that the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host MemorySpace (HMS), (ii) the Decomposition Submodule No. 2 401″ andDistribution Module 402″ are realized (as a graphics hub) in on the dieof a hybrid CPU/GPU fusion-architecture chip within the host computingsystem and having multiple GPUs 1242″ driven with one or more GPUs on anexternal graphics card 205 (connected to the CPU/GPU chip) in aparallelized manner under the control of the AMCM, (iii) theRecomposition Module 403″ is implemented across two or more GPUs 715,716 provided on the CPU/GPU fusion chip die and external graphics cards,(iv) the Decomposition Submodule No. 1 transfers graphics commands anddata (GCAD) to the Decomposition Submodule No. 2, (v) the DecompositionSubmodule No. 2 divides (i.e. splits up) the stream of graphic commandsand data (GCAD) according to the required parallelization mode,operative at any instant in time, (vi) the Distribution Module 402″ usesthe crossbar switch (i.e. interconnect) on the processor die todistribute the graphic commands and data (GCAD) to the internal GPUs andexternal GPUs, (vii) the Recomposition Module 403″ transfers compositedpixel data (CPD) between the GPUs during the image recomposition stage,and (viii) finally recomposited pixel data sets are displayed asgraphical images on one or more display devices 106 connected to theexternal graphics card connected to the hybrid CPU/GPU chip via aPCI-express interface;

FIG. 7B 9-1 is a schematic representation of a twentieth illustrativeembodiment of the MMPGRS of the present invention, following theMulticore CPU Class of MMPGRS Architecture described in FIG. 7A 1-2, andshowing (i) that the Automatic Mode Control Module (AMCM) 400 and theDecomposition Submodule No. 1 401′ reside as a software package in theHost Memory Space (HMS) while the Decomposition Submodule No. 2 401″,Distribution Module 402″ and Recomposition Module 403″ are realized (asa graphics hub) in on the die of a multi-core CPU chip within the hostcomputing system and having multiple CPU cores, some of which implementmultiple soft parallel graphics pipelines (“soft GPUs”) driven in aparallelized manner under the control of the AMCM, (ii) theDecomposition Submodule No. 1 transfers graphics commands and data(GCAD) to the Decomposition Submodule No. 2 via the North memory bridgecircuit and interconnect network within the multi-core CPU chip, (iii)the Decomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iv) theDistribution Module 402″ uses the crossbar switch (i.e. interconnect) onthe processor die to distribute the graphic commands and data (GCAD) tothe multiple soft parallel graphics pipelines (implemented by themultiple CPU cores), (v) the Recomposition Module 403″ transferscomposited pixel data (CPD) between the multiple CPU cores during theimage recomposition stage, and (vi) finally recomposited pixel data setsare displayed as graphical images on one or more display devices 106connected to the North memory bridge chip via a display interface;

FIG. 7B 9-2 is a schematic representation of a twenty-first illustrativeembodiment of the MMPGRS of the present invention, following theMulticore CPU Class of MMPGRS Architecture described in FIG. 7A 1.2, andshowing (i) that the Automatic Mode Control Module (AMCM) 400 and theDecomposition Submodule No. 1 401′ resides as a software package in theHost Memory Space (HMS) while the Decomposition Submodule No. 2 401″,the Distribution Module 402″ and the Recomposition Module 403″ arerealized as a graphics hub chip within a gaming console systeminterconnecting a multi-core CPU ship and a cluster of GPUs on the gameconsole board, so that the GPUs are driven in a parallelized mannerunder the control of the AMCM, (ii) the Decomposition Submodule No. 1transfers graphics commands and data (GCAD) to the DecompositionSubmodule No. 2 via the interconnects within the North memory bridgechip and the multi-core CPU chip, (iii) the Decomposition Submodule No.2 divides (i.e. splits up) the stream of graphic commands and data(GCAD) according to the required parallelization mode, operative at anyinstant in time, (iv) the Distribution Module 402″ uses the interconnect(i.e. crossbar switch) in the multi-core CPU chip to distribute thegraphic commands and data (GCAD) to the multiple soft graphics pipelines(e.g. soft GPUs) and the GPUs on the external graphics card 205, and (v)the Recomposition Module 403″ transfers composited pixel data (CPD)between the soft graphics pipelines on the multi-core CPU chip and hardGPUs on the external graphics card during the image recomposition stage,and (vi) finally recomposited pixel data sets are displayed as graphicalimages on one or more display devices 106 connected to the externalgraphics card which is connected to the multi-core CPU chip via aPCI-express interface;

FIG. 7B 10 is a schematic representation of a twenty-second illustrativeembodiment of the MMPGRS of the present invention, following the GameConsole Class of MMPGRS Architecture described in FIG. 7A 1-2, andshowing (i) that the Automatic Of Mode Control Module (AMCM) 400 and theDecomposition Submodule No. 1 401′ are realized as a software package711 within the Host Memory Space (HMS), while the DecompositionSubmodule No. 2 401″, the Distribution Module 402″ and the RecompositionModule 403′ are realized as a graphics hub semiconductor chip within thegame console system in which multiple GPUs are driven in a parallelizedmanner under the control of the AMCM, (ii) the Decomposition SubmoduleNo. 1 transfers graphics commands and data (GCAD) to the DecompositionSubmodule No. 2, via the memory controller on the multi-core CPU chipand the interconnect in the graphics hub chip of the present invention,(iii) the Decomposition Submodule No. 2 divides (i.e. splits up) thestream of graphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iv) theDistribution Module 402″ distributes the graphic commands and data(GCAD) to the multiple GPUs, (v) the Recomposition Module 403″ transferscomposited pixel data (CPD) between the multiple GPUs during the imagerecomposition stage, and (vi) finally recomposited pixel data sets(recomposited within the vertex and/or fragment shaders of the primaryGPU) are displayed as graphical images on one or more display devices106 connected to the primary GPU 715 via a analog display interface;

FIG. 7B 11 is a schematic representation of a twenty-third illustrativeembodiment of the MMPGRS of the present invention, following the GameConsole Class of MMPGRS Architecture described in FIG. 7A 1-2, andshowing (i) that the Automatic Mode Control Module (AMCM) 400 and theDecomposition Submodule No. 1 401′ are realized as a software package711 within the Host Memory Space (HMS) of the host computing systemwhile the Decomposition Submodule No. 2 401″ and Distribution Module402′ are realized as a graphics hub semiconductor chip within the gameconsole system in which multiple GPUs are driven in a parallelizedmanner under the control of the AMCM, (ii) the Decomposition SubmoduleNo. 1 transfers graphics commands and data (GCAD) to the DecompositionSubmodule No. 2, via the memory controller on the multi-core CPU chipand the interconnect in the graphics hub chip of the present invention,(iii) the Decomposition Submodule No. 2 divides (i.e. splits up) thestream of graphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iv) theDistribution Module 402′ distributes the graphic commands and data(GCAD) to the multiple GPUs, (v) the Recomposition Module 403′, realizedprimarily within the substructure of the primary GPU, transferscomposited pixel data (CPD) between the multiple GPUs during the imagerecomposition stage, and (vi) finally recomposited pixel data sets(recomposited within the vertex and/or fragment shaders of the primaryGPU) are displayed as graphical images on one or more display devices106 connected to the primary GPU 715 via an analog display interface orthe like;

FIG. 8A is a schematic block representation of an illustrativeimplementation of the MMPGRS of the present invention following the HubClass of MMPGRS Architecture described in FIG. 7A 1-2, wherein (i) theAMCM and Decomposition No. 1 Module are implemented as asoftware-package 701 within host memory space (HMS) of the hostcomputing system, (ii) multiple discrete graphic cards are connected tothe bridge circuit of the host computing system by way of ahardware-based graphics hub chip of the present invention 404″, 402″,403″, 404″, (iii) hardware-based Distribution and Recomposition Modules402″ and 403″ are realized on the hardware-based graphics hub chip ofthe present invention, and (iv) a graphics display device is connectedto the primary GPU;

FIG. 8A 1 is a schematic representation of a first illustrativeembodiment of the MMPGRS implementation of FIG. 8A, showing a possiblepackaging of the Hub architecture of the present invention as anassembly comprising a Hub-extender card 811 carrying multiple (e.g.dual) graphics cards 812, 813 supported on a motherboard 814 within thehost computing system;

FIG. 8A 2 is a schematic representation of a second illustrativeembodiment of the MMPGRS implementation of FIG. 8A, showing a possiblepackaging of the Hub architecture of the present invention as anexternal box containing a Hub chip of the present invention mounted on aPC board, that is connected to the motherboard of the host computingsystem via a wire harness or the like, and supporting a plurality ofgraphics cards 813 that are connected to the Hub chip;

FIG. 8A 3 is a schematic representation of a third illustrativeembodiment of the MMPGRS implementation of FIG. 8A, showing a possiblepackaging of the Hub architecture of the present invention realized as agraphics Hub chip of the present invention mounted on the motherboard814 of the host computing system, which supports multiple graphics cards813 with multiple GPUs;

FIG. 8B is a schematic block representation of an illustrativeimplementation of the MMPGRS of the present invention following theHub+GPU-Recomposition Class of MMPGRS Architecture described in FIG. 7A1-2, wherein (i) the AMCM and Decomposition No. 1 Submodule areimplemented as a software-package 701 within host memory space (HMS) ofthe host computing system, (ii) multiple discrete graphic cards areconnected to a bridge chipset on the host computing system by way of ahardware-based graphics hub chip realizing the Decomposition No. 2Submodule 401″ and the Distribution Module 402″, (iii) the RecompositionModule 403″ is implemented across two or more GPUs 715, 716, and (iv) agraphics display device is connected to the primary GPU;

FIG. 8B 1 is a schematic representation of a first illustrativeembodiment of the MMPGRS implementation of FIG. 8B, showing a possiblepackaging of the Hub+GPU Recomposition architecture of the presentinvention as an assembly comprising a graphic hub-extender card 811carrying multiple (e.g. dual) graphics cards 812, 813 supported on amotherboard 814 within the host computing system;

FIG. 8B 2 is a schematic representation of a second illustrativeembodiment of the MMPGRS implementation of FIG. 8B, showing a possiblepackaging of the Hub architecture of the present invention as anexternal box containing a Hub chip of the present invention mounted on aPC board, that is connected to the motherboard of the host computingsystem via a wire harness or the like, and supporting a plurality ofgraphics cards 813 that are connected to the graphics hub chip;

FIG. 8B 3 is a schematic representation of a third illustrativeembodiment of the MMPGRS implementation of FIG. 8B, showing a possiblepackaging of the Hub architecture of the present invention realized as agraphics hub chip of the present invention mounted on the motherboard814 of the host computing system, which supports multiple graphics cards813 with multiple GPUs;

FIG. 8C is a schematic block representation of an illustrativeembodiment of the MMPGRS of the present invention following the HM Classof MMPGRS Architecture described in FIG. 7A 1-2, wherein (i) the AMCM,Decomposition, Distribution and Recomposition Modules are implemented asa software-package 701 within host memory space (HMS) of the hostcomputing system, (ii) multiple discrete GPUs on one or more graphicscards, are connected to the bridge circuit on the host computing system,and (iii) a graphics display device is connected to the primary GPU;

FIG. 8C 1 is a schematic representation of a first illustrativeembodiment of the MMPGRS implementation of FIG. 8C, wherein discretemultiple graphics cards 851, each supporting at least a single GPU, areinterfaced with the bridge circuit chipset of the CPU motherboard by wayof a PCI-express or like interface;

FIG. 8C 2 is a schematic representation of a second illustrativeembodiment of the MMPGRS implementation of FIG. 8C, wherein multipleGPUs are realized on a single graphics card 852 which is interfaced tobridge circuit on the CPU motherboard by way of a PCI-express or likeinterface;

FIG. 8C 3 is a schematic representation of a third illustrativeembodiment of the MMPGRS implementation of FIG. 8C, wherein multiplediscrete graphics cards 851, each supporting at least a single GPU, areinterfaced with the bridge circuit on a board within an external box 821that is interface to the motherboard within the host computing system;

FIG. 8D is a schematic block representation of an illustrativeembodiment of the MMPGRS of the present invention following theHub+GPU-Recomposition Class of MMPGRS Architecture described in FIG. 7A1-2, wherein (i) the AMCM, Decomposition Submodule No. 1 and aDistribution Module are implemented as a software-package 701 withinhost memory space (HMS) of the host computing system, (ii) multiplediscrete GPUs on one or more external graphics cards are connected tothe bridge circuit of the host computing system, (iii) a RecompositionModule 403″ is implemented across two or more GPUs 715, 716, and (iv) agraphics display device is connected to the primary GPU;

FIG. 8D 1 is a schematic representation of a first illustrativeembodiment of the MMPGRS implementation of FIG. 8D, wherein discretemultiple graphics cards 851, each supporting at least a single GPU, areinterfaced with the bridge circuit chipset of the CPU motherboard by wayof a PCI-express or like interface;

FIG. 8D 2 is a schematic representation of a second illustrativeembodiment of the MMPGRS implementation of FIG. 8D, wherein multipleGPUs are realized on a single graphics card 852 which is interfaced tobridge circuit on the CPU motherboard by way of a PCI-express or likeinterface;

FIG. 8D 3 is a schematic representation of a third illustrativeembodiment of the MMPGRS implementation of FIG. 8D, wherein multiplediscrete graphics cards 851, each supporting at least a single GPU, areinterfaced with the bridge circuit on a board within an external box 821that is interface to the motherboard within the host computing system;

FIG. 9A is a schematic block representation of an illustrativeimplementation of the MMPGRS of the present invention following the HubClass of MMPGRS Architecture described in FIG. 7A 1-2, wherein (i) theAMCM and Decomposition Submodule No. 1 are realized as a softwarepackage 711 on the host memory space (HMS), (ii) multiple GPUs (i.e.Primary GPU 715 and Secondary GPUs 716) are assembled on a externalgraphics card 902 which connects the GPUs to the bridge circuit on thehost computing system by way of a hardware-based graphics hub chipimplementing the Decomposition Submodule No. 2 401″, the DistributionModule 402″ and the Recomposition Module 403″, and (iii) a graphicsdisplay device is connected to the primary GPU;

FIG. 9A 1 is a schematic representation of an illustrative embodiment ofthe MMPGRS of FIG. 9A, wherein multiple GPUs 715, 716 and hardware-basedDecomposition Submodule No. 2 401″, Distribution Module 402″ and theRecomposition Modules 403″ are implemented as a graphics hub chip orchipset 401″, 402′, 403,″ and 404″ on a single graphics display card902, which is interfaced to the bridge circuit on the motherboard 814within the host computing system;

FIG. 10A is a schematic block representation of an illustrativeimplementation of the MMPGRS of the present invention following the HubClass of MMPGRS Architecture described in FIG. 7A 1-2, wherein (i) theAMCM and Decomposition Submodule No. 1 are realized as a softwarepackage 711 on the host memory space (HMS), (ii) a single SOC-basedgraphics chip 1001 mounted on a single graphics card 1002 is interfacedwith a bridge circuit on the motherboard 1002, and supporting multipleGPUs (i.e. the primary GPU and secondary GPUs), (iii) hardware-basedDecomposition Submodule No. 2, the Distribution Module and theRecomposition Module are implemented on the SOC-based graphics chip1001, and (iv) a graphics display device is connected to the primaryGPU;

FIG. 10A 1 is a schematic representation of possible packaging of theSOC-based graphics hub chip 101 depicted in FIG. 10A, wherein multipleGPUs 715, 716 and hardware-based Decomposition Submodule 401″,Distribution Module 402″, and Recomposition Module 4-3″ are realized ona single SOC implementation 1001 mounted on a single graphics card 1002;

FIG. 10B is a schematic block representation of an illustrativeimplementation of the MMPGRS of the present invention following theHub+GPU-Recomposition Class of MMPGRS Architecture described in FIG. 7A1-2, wherein (i) the AMCM and Decomposition Submodule No. 1 are realizedas a software package 711 on the host memory space (HMS), (ii) a singleSOC-based graphics chip 1003 mounted on a single graphics card 1002 isinterfaced with a bridge circuit on the motherboard 1002, and supportingmultiple GPUs (i.e. the primary GPU and secondary GPUs), (iii)hardware-based Decomposition Submodule No. 2 and the Distribution Moduleare implemented on the SOC-based graphics hub chip 1001, (iv) theRecomposition Module is implemented across two or more GPUs 715, 716,and (v) a graphics display device is connected to the primary GPU by wayof a display interface implemented on the SOC-based graphics hub chip;

FIG. 10B 1 is a schematic representation of possible packaging of theSOC-based graphics hub chip 101 depicted in FIG. 10B, wherein multipleGPUs 715, 716 and hardware-based Decomposition Submodule 401″,Distribution Module 402″, and Recomposition Module 4-3″ are realized inthe primary GPU of a single SOC implementation 1003 mounted on a singlegraphics card 1002;

FIG. 10C is a schematic block representation of an illustrativeimplementation of the MMPGRS of the present invention following theHMS+GPU-Recomposition Class of MMPGRS Architecture described in FIG. 7A1-2, wherein (i) the AMCM, Decomposition Module and Distribution Moduleare realized as a software package 701 on the host memory space (HMS),(ii) a single multi-GPU chip 1031 mounted on a single graphics card 1002is interfaced with a bridge circuit on the motherboard, and supportingmultiple GPUs (i.e. the primary GPU and secondary GPUs), (iii) theRecomposition Module is implemented across two or more GPUs 715, 716,and (iv) a graphics display device is connected to the primary GPU byway of a display interface implemented on the multi-GPU chip;

FIG. 10C 1 is a schematic representation of possible packaging of themulti-GPU chip 1031 depicted in FIG. 10C, wherein multiple GPUs 715, 716and Recomposition Module/Process 403″ are implemented in the primary GPU715 of a multi-GPU chip 1031;

FIG. 11A is a schematic block representation of an illustrativeimplementation of the MMPGRS following the Chipset Class of MMPGRSArchitecture described in FIG. 7A 1-2, wherein (i) the AMCM andDecomposition Submodule No. 1 are realized as a software package 711within the host memory space (HMS) of the host computing system, (ii)plurality of GPUs 852 on one or more external graphics cards 851 isconnected to the bridge circuit on the host computing platform, (iii) anintegrated graphics device (IGD) 1101, supporting hardware-basedDecomposition Submodule No. 2, the Distribution Module 402″ andRecomposition Module 403″, are implemented within the bridge circuit1101 on the motherboard 814 of the host computing system, and (iv) adisplay device is interfaced to the primary GPU by way of a PCI-expressinterface or the like;

FIG. 11A 1 is a schematic representation of a first illustrativeembodiment of the Chipset MMPGRS implementation of FIG. 11A, whereinmultiple discrete graphics cards 851, each supporting at least a singleGPU, are interfaced with the bridge Circuit on a board within, anexternal box 821 that is interface to the motherboard within the hostcomputing system;

FIG. 11A 1 is a schematic representation of a second illustrativeembodiment of the Chipset MMPGRS implementation of FIG. 11A, whereindiscrete multiple graphics cards 851, each supporting at least a singleGPU, are interfaced with the bridge circuit chipset of the CPUmotherboard by way of a PCI-express or like interface;

FIG. 11A 3 is a schematic representation of a third illustrativeembodiment of the Chipset MMPGRS implementation of FIG. 11A, whereinmultiple GPUs are realized on a single graphics card 852 which isinterfaced to bridge circuit on the CPU motherboard by way of aPCI-express or like interface;

FIG. 11B is schematic representation of an illustrative implementationof the MMPGRS following the CPU/GPU Fusion Class of MMPGRS Architectureor Multi-Core Class MMPGRS Architecture described in FIG. 7A 1-2,wherein (i) a CPU/GPU fusion-architecture chip or a multi-core CPU chipis mounted on the motherboard of a host computing system having memoryand North and South bridge circuits, (ii) the software-based AMCM andDecomposition Submodule No. 1 are realized as a software package 701within the host memory space (HMS) of the host computing system whileDecomposition Submodule No. 2, the Distribution Module and theRecomposition Module are realized on the die of the CPU/GPUfusion-architecture chip or the multi-core CPU chip, and (iii) multipleGPUs on external graphic cards or elsewhere, are interfaced to theCPU/GPU fusion-architecture chip or the multi-core CPU chip, by way of aPCI-express or like interface, and (iv) a display device is interfacedto the primary GPU by way of a PCI-express interface or the like.

FIG. 11B 1 is a schematic representation of a first illustrativeembodiment of the CPU/GPU Fusion or Multi-Core MMPGRS implementation ofFIG. 21B, wherein a CPU/GPU Fusion or Multi-Core chip is used to drivean assemble of graphic cards or GPUs on one or more external graphicscards 851;

FIG. 11B 2 is a schematic representation of a second illustrativeembodiment of the Chipset MMPGRS implementation of FIG. 11B, wherein aCPU/GPU Fusion or Multi-Core chip is used to drive an assemble of GPUson a single external graphics card 852;

FIG. 11B 3 is a schematic representation of a third illustrativeembodiment of the Chipset MMPGRS implementation of FIG. 11B, wherein aCPU/GPU Fusion or Multi-Core chip is used to drive only an assemble ofinternal GPUs on the CPU/GPU Fusion or Multi-Core chip;

FIG. 11C is schematic representation of an illustrative implementationof the MMPGRS following the Game Console Class of MMPGRS Architecturedescribed in FIG. 7A 1-2, wherein (i) the AMCM 400 and DecompositionSubmodule No. 1 401′ are realized as a software package within the hostmemory space (HMS) of the game console system, (ii) a graphics hub chip401″, 402″, 403″, 404″ mounted on the PC board of the game consolesystem implements the Decomposition Submodule No. 2 401″, theDistribution Module 402′, the Recomposition Module 403′ as well as aninterconnect network (e.g. crossbar switch) 404″, (iii) multiple GPUs onthe PC board of the game console system are interfaced to Distributionand Recomposition Modules by way of the interconnect 404″ within thegraphics hub chip, and optionally, the Recomposition Module can beimplemented within two or more GPUs 715, 716, and (iv) a display device106 is interfaced to the primary GPU by way of an analog displayinterface or the like;

FIG. 11C 1 is a schematic representation of an illustrative embodimentof the Game Console MMPGS implementation of FIG. 11D, showing itscontroller in combination with its game console unit;

FIG. 12A is a schematic representation of a multi-user computer networksupporting a plurality of client machines, wherein one or more clientmachines (i) employ the MMPGRS of the present invention following anyMMPGRS Architecture described in FIG. 7A 1-2, and (ii) respond touser-system interaction input data streams from one or more networkusers who might be local to each other as over a LAN, or be remote toeach other, as when operating over a WAN or the Internet infrastructure;and

FIG. 12B is a schematic representation of a multi-user computer networksupporting a plurality of client machines, wherein one or more clientmachines (i) employ the MMPGRS of the present invention following anyMMPGRS Architecture described in FIG. 7A 1-2, and (ii) respond touser-system interaction input data streams from one or more networkusers who might be local to each other as over a LAN, or be remote toeach other, as when operating over a WAN or the Internet infrastructure.

DETAILED DESCRIPTION OF THE ILLUSTRATIVE EMBODIMENTS OF THE PRESENTINVENTION

Referring now to FIGS. 4A through 12B in the accompanying Drawings, thevarious illustrative embodiments of the Multi-Mode Parallel GraphicsRendering System (MMPGRS) and Multi-Mode Parallel Graphics RenderingProcess (MMPGRP) of the present invention will now be described in greattechnical detail, wherein like elements will be indicated using likereference numerals.

In general, one aspect of the present invention teaches how todynamically retain high and steady performance of a three-dimensional(3D) graphics system on conventional platforms (e.g. PCs, laptops,servers, etc.), as well as on silicon level graphics systems (e.g.graphics system on chip (SOC) implementations, integrated graphicsdevice IGD implementations, and hybrid CPU/GPU die implementations).This aspect of the present invention is accomplished by means of a novelarchitecture supporting adaptive graphics parallelism having bothsoftware, hardware and hybrid embodiments.

The MMPGRS and MMPGRP of the present invention fulfills a great need inthe marketplace by providing a highly-suited parallelism scheme, Byvirtue of the present invention, different GPPL-based parallel renderingschemes, supported on the MMPGRS, dynamically alternate throughout thecourse of any particular graphics application running on the host systemin which the MMPGRS is embodied, and adapting the optimal parallelrendering method (e.g. Image/Frame, Time or Object Division) inreal-time to meet the changing needs of the graphics application(s).

The MMPGRS of the Present Invention Employs Automatic Mode and ControlModule (AMCM)

FIG. 4A shows the MMPGRS of the present invention employing automatic 3Dscene profiling and multiple GPPL control, and supporting at least threedifferent parallelization modes (e.g. Image/Frame, Time and ObjectDivision). As shown, the MMPGRS comprises two primary subcomponents,namely:

(1) Multi-Mode Parallel Graphics Rendering Subsystem 420 including (i) aDecomposition Module 401, Distribution Module 402 and RecompositionModule 403 for supporting three stages of parallelization namelydecomposition, distribution, and recomposition, and (ii) an Array orCluster of Graphic Processing Pipelines (GPPLs) for supporting anddriving Graphics Rendering and Image Display Processes; and

(2) a Automatic Mode Control Module (AMCM) 400, described in FIGS. 4Cthrough 5C4 and 6A, for dynamically profiling Graphics-basedApplications running on the host computing system, and controlling thevarious modes of parallelism supported by the MMPGRS of the presentinvention.

In general, the GPPLs can be realized in various ways, including (i)Graphic Processing Units (GPUs) 407 as shown in FIGS. 4B1 and 4B2,and/or (ii) Computational Processing Units (CPUs), or CPU-cores, asshown in FIGS. 4B3 and 4B4.

As shown in FIGS. 4A and 4D, the Graphics Commands and Data (CGAD) tothe MMPGRS will typically be produced and provided from theGraphics-based Application being executed by one or more CPUs andassociated memory on the host computing system. In contrast, theInteraction Data will be supplied from the user or users interactionwith the host computing system.

In general, the host computing system may be a PC-level computer,application server, laptop, game console system, portable computingsystem, or the like supporting the real-time generation and display of3D graphics), and that the MMPGRS may be embodied within any such systemin accordance with the principles of the present invention.

The Graphics Processing Pipelines (GPPLs) Employed within the MMPGRS ofthe Present Invention

In general, each GPPL employed within the MMPGRS of the presentinvention can be realized in a variety of different ways. However, ingeneral, each graphics processing pipeline (GPPL) will typically includesome basic structures including for example, video memory and acomputational unit such as a GPU, or CPU having multi-cores typicallyimplementing SIMD elements. When using GPUs, the graphic processingpipelines (GPPLs) are often considered “hard” graphical processingpipelines. When using CPUs, the graphic processing pipelines are oftenconsidered “soft” graphical processing pipelines. In either case, eachgraphic processing pipeline (GPPL) provides sufficient computational andmemory/buffering resources to carry out the execution of graphicscommands and the processing of graphics data, as specified by thegraphical rendering processed required by the graphics-based Applicationrunning on the host computing system, at any particular instant in time.

In FIGS. 4B1 and 4B2, two illustrative embodiments for the GPU-basedgraphics processing pipeline approach are shown. In FIG. 4B 3, oneillustrative embodiment is shown for the CPU-based graphics processingpipeline approach.

As shown in FIG. 4B 1, each GPU-based graphics processing pipeline(GPPL) deployed in the MMPGRS of a first illustrative embodimentcomprises: (i) video memory (e.g. a stencil memory buffer, a depthmemory buffer, and a color memory buffer); and (ii) a classicshader-based GPU which includes: a geometry subsystem; a set up engine;and a pixel subsystem. As shown, the geometry subsystem furthercomprises a vertex shader which implements a graphics processingfunction that perform is 3D geometrical transformations and lightingcalculations on the objects' vertex data. The Setup engine assemblesprimitives (lines, points, triangles) from vertices, assigns parametersto primitives, divides the primitives into tiles, and distributes thesetiles to the pixel pipelines of the Pixel subsystem. The Pixel subsystemfurther comprises: a pixel shader for receiving input from the Setupengine and the video memory and performing shading and texturing ofpixels; and a plurality of raster operators which receive output fromthe pixel shader and produce blending, z-buffering and antialiasing ofpixels, storing them into Frame Buffer buffer. This graphics pipelinearchitecture can be found used in conventional graphics devices such asnVidia's GeForce 7700.

As shown in FIG. 4B 2, each GPU-based graphics processing pipeline(GPPL) deployed in the MMPGRS of a second illustrative embodimentcomprises: (i) video memory (e.g. a stencil memory buffer, a depthmemory buffer, and a color memory buffer); and (ii) a shader-based GPUwhich includes: a geometry subsystem; a rasterizer; and a pixelsubsystem. As shown, the geometry subsystem further comprises: an inputassembler for gathering vertex data from the CPU and converting itsformat, and generating various index 1Ds that are helpful for performingvarious repeated operations on vertices, primitives, and scene objects;a vertex shader for performing 3D geometrical transformations andlighting calculations on the objects' vertex data; and a geometry shaderpermitting a range of effects and features, such as process entireprimitives as inputs and generate entire primitives as output, ratherthan processing just one vertex at a time, as with a vertex shader,while reducing dependence on the CPU for geometry processing. The streamoutput permits data generated from geometry shaders to be forwarded backto the top of the pipeline to be processed again. The rasterizerassembles primitives (lines, points, triangles) from vertices, assignsparameters to primitives, and converts them into pixels for output tothe Pixel subsystem. The pixel subsystem further comprises: a pixelshader for receiving input from the Setup engine and the video memoryand performing shading and texturing of pixels; and a plurality ofraster operators which receive output from the pixel shader and produceblending, z-buffering and anti-aliasing of pixels, storing them out intoFrame Buffer (FB). This graphics pipeline architecture can be found usedin conventional graphics devices such as nVidia's GeForce 8800 GTX.

As shown in FIG. 4B 3, each CPU-based graphics processing pipeline(GPPL) deployed in the MMPGRS of a third illustrative embodimentcomprises: (i) a video memory structure supporting a frame buffer(including stencil, depth and color buffers); (ii) a memory controller;(iii) a graphics processing pipeline realized by one cell of amulti-core CPU chip, consisting of 16 in-order SIMD processors; (iv) L2cache memory; and (v) a GPU-specific extension, namely, a texturesampler, for loading texture maps from memory, filtering them forlevel-of-detail, and feeding the same to the pixel processing portion ofthe graphic processing pipeline (GPPL). This graphics pipelinearchitecture can be found used in such conventional devices as Larrabeemulti-core processor by Intel.

Notably, as shown in FIG. 4A, while the array of GPPLs 407 comprises Npairs of GPU or CPU and Video Memory pipelines, only one GPPL in thearray, termed “primary GPPL,” is responsible for driving the displayunit which may be realized as a LCD panel, an LCD or DLP Image/Video“Multi-Media” Projector, or the like. All other GPPLs in the array aredeemed “secondary GPPLs.”

The Multi-Mode Parallel Graphics Rendering Subsystem

In the Multi-Mode Parallel Graphics Rendering Subsystem 420, each stage(or Module) is induced or set up into a sub-state by a set of parametersmanaged within the MMPGRS, namely: parameter A for Module 401; parameterB for Module 402; and parameter C for Module 403. The state ofparallelism of the overall MMPGRS is established by the combination ofsub-state parameters A, B and C, as listed in the Mode/State DefinitionTable of FIG. 4C, which will be elaborated hereinafter.

The unique flexibility of the Multi-Mode Parallel Graphics RenderingSubsystem 420 stems from its ability to quickly change its sub-states,resulting in transition of the overall graphic system (i.e. MMPGS) toanother parallel state of operation, namely: the Object Division State,the Image Division State or the Time Division State, as well as to otherpotential parallelization schemes that may be developed and readilyprogrammed into the MMPGRS platform of the present invention.

Implementing Parallelization Modes Through a Net Combination ofSub-States (A:B:C) Among the Decomposition, Distribution AndRecomposition Modules

As indicated in the State Table of FIG. 4C, the net combination of allSub-States (A:B:C) among the Decomposition Module 401, DistributionModule 402 and Recomposition Module 403, respectively, implements thevarious parallelization schemes (i.e. parallelization modes) supportedon the MMPGRS of the present invention, which will now be describedhereinbelow. Thus, the Decomposition Module 401, Distribution Module 402and Recomposition Module 403 cooperate to carry out all functionsrequired by the different parallelization schemes supported on theMMPGRS platform of the present invention. It is appropriate at thisjuncture to described how the primary modes of parallelism (i.e. Image,Time and Object Division) are implemented on the MMPGS usingcombinations of sub-state parameters (A:B:C).

The Image Division State of Parallel Operation:

In the Image Division State of Operation, the Decomposition Module 401is set to the Image Decomposition Sub-State or Sub-mode (A=2),multiplicating the same command and data stream to all GPUs, anddefining unique screen portion for each one, according to the specificImage Division Mode in use (e.g. split screen, or tiled screen). TheDistribution Module is set in Broadcast Sub-mode B=2, to physicallybroadcast the stream to all GPUs. Finally, the Recomposition Module No.1 set to Screen-based Sub-mode C=2, and collects all the partial imagesinto final frame buffer, performing the screen based composition.

The Time Division State of Parallel Operation:

In the Time Division State of Operation, each GPU renders the nextsuccessive frame. The Decomposition Module is set to the AlternateSub-mode, A=3, alternating the command and data stream among GPUs onframe basis. The Distribution Module is set to the Single Sub-mode, B=3,physically moving the stream to the designated GPU. Finally, theRecomposition Module is set to None, C=3, as no merge is needed and theframe buffer is just moved from the designated GPU to the screen fordisplay.

The Object Division State of Parallel Operation:

In the Object Division State of operation, the Decomposition Module isset to the Object Decomposition Sub-mode, A=1, decomposing the commandand data stream, and targeting partial streams to different GPUs. TheDistribution Module is set to the Divide Sub-mode, B=1, physicallydelivering the partial commands and data to GPUs. Finally theRecomposition Module is set to Test-Based Sub-mode, C=1, compositing theframe buffer color components of GPUs, based on depth and/or stenciltests.

The Single GPPL State of Operation:

While the Single GPPL State of Operation is a non-parallel state ofoperation, it is allowed and supported in the system of the presentinvention as this state of operation is beneficial in some exceptionalcases. In the Single GPPL State, the Decomposition, Distribution, andRecomposition Modules are set on Single (A=4), Single (B=3) and None(C=3), respectively. Only one GPPL, among all pipelines supported by theMMPGRS, is used in the single GPPL state of operation.

Description of the Decomposition Module of the MMPGRS of the PresentInvention

The primary function of the Decomposition Module 401 is to divide (i.e.split up) the stream of graphic commands and data (GCAD) according tothe required parallelization mode, operative at any instant in time. Ingeneral, the typical graphic processing s pipeline is fed by stream ofgraphic commands and data from the application and graphics library(OpenGL or Direct 3D). This stream, which is sequential in nature, hasto be properly handled and eventually partitioned, according toparallelization mode (i.e. method) used. Under the AMCM 400, theDecomposition Module can be set to different decomposing sub-states (A=1through A=4), according to FIG. 4C, namely: Object DecompositionSub-state A=1 during the Object Division State; Image DecompositionSub-state A=2 during the Image Division State; Alternate DecompositionSub-state A=3 during the Time Division State; and the Single Sub-stateA=4 during the Single GPPL (Non-Parallel) State. Each one of theseparallelization states (i.e. Object, Image, Time and Single/Non-ParallelStates) will be described in great technical detail below.

As shown in FIG. 4A, the Decomposition Module 401 is preferablyimplemented using two submodules, namely: (i) a Decomposition SubmoduleNo. 1 including an OS-GPU Interface and Utilities Module; and (ii) aDecomposition Submodule No. 2, including a Division Control Module and aState Monitoring Module. The subcomponents of these submodules will bedescribed in detail below.

The OS-GPU Interface and Utilities Module

The OS-GPU interface and Utilities Module performs all the functionsassociated with interaction with the Operating System (OS), GraphicsLibrary (e.g. OpenGL or DirectX), and interfacing with GPUs orCPU-cores, as the case may be. The OS-GPU interface and Utilities Moduleis responsible for interception of the graphic commands from thestandard graphic library, forwarding and creating graphic commands tothe Vendor's GPU Driver, controlling registry, installations, OSservices and utilities. Another task performed by this module is readingPerformance Data from different sources (e.g. GPUs, vendor's driver, andchipset) and forwarding the Performance Data to the Automatic ModeControl Module (AMCM). Also, the OS-GPU Interface and Utilities Moduleincludes software drivers that drive subcomponents within theDecomposition, Distribution and/or Recomposition Modules that areimplemented in system architectures (e.g. Hub, Chipset, etc identifiedin FIG. 4A 1-2 and shown in FIGS. 7B1 through 7B11) in which both theDecomposition and Distribution Modules are not implemented as softwarepackages within the Host Memory Space (HMS) of the host computing systemin which the MMPGRS is embodied.

The Division Control Module

In the Division Control Module, all graphics commands and data areprocessed for decomposition and marked for division. However, thesecommands and data are sent in a single stream into the DistributionModule for physical distribution. The Division Control Module controlsthe division parameters and data to be processed by each GPU, accordingto parallelization scheme instantiated at any instant of systemoperation (e.g. division of data among GPUs in the Object Division Mode,or the partition of the image screen among GPUs in the Image DivisionMode).

In the Image Division Mode, the Division Control Module assigns forduplication all the geometric data and common rendering commands to allGPUs. However specific rendering commands to define clipping windowscorresponding to image portions at each GPU, are assigned separately toeach GPU.

In the Object Division Mode, polygon division control involves sendingeach polygon (in the scene) randomly to a different GPU within theMMPGRS. This is an easy algorithm to implement, and it turns out to bequite efficient. There are different variations of this basic algorithm,as described below.

Polygon Division Control By Distribution Of Vertex Arrays

According to this method, instead of randomly dividing the polygons, thevertex-arrays can be maintained in their entirety and sent to differentGPUs, as the input might be in the form of vertex arrays, and dividingit may be too expensive.

Polygon Division Control by Dynamic Load Balancing

According to this method, GPU loads are detected at real time and thenext polygon is sent to the least loaded GPU. Dynamic load balancing isachieved by building complex objects (out of polygons). GPU loads aredetected at real time and the next object is sent to the least loadedGPU.

Handling State Validity Across The MMPGRS By State Monitoring

The graphic libraries (e.g. OpenGL and DirectX) are state machines.Parallelization must preserve a cohesive state across all of the GPUpipelines in the MMPGRS. According to this method, this is achieved bycontinuously analyzing all incoming graphics commands, while the statecommands and some of the data is duplicated to all graphics pipelines inorder to preserve the valid state across all of the graphic pipelines inthe MMPGRS. This function is exercised mainly in Object Division Mode,as disclosed in detail in Applicant's previous International PatentPCT/IL04/001069, now published as WIPO International Publication No. WO2005/050557, incorporated herein by reference in its entirety.

The Description of the Distribution Module of the Present Invention

The primary function of the Distribution Module 402 is to physicallydistribute the streams of graphics data and commands to the cluster ofGPUs supported on the MMPGRS platform. Under the AMCM 400, theDistribution Module can be set to different distribution sub-states (B=1through B=3), according to FIG. 4C, namely: the Divide Sub-state B1during the Object Division State; the B=2 Sub-state (i.e. the BroadcastSub-state) during the Image Division State; and the B3 Sub-state (i.e.Single GPU Sub-state) during the Time Division and Single GPU (i.e.Non-Parallel system) States. As shown in FIG. 4A, an additional sourceof Performance Data (i.e. beyond the GPUs, vendor's driver, and chipset)includes the internal Profiler employed in the Distribution Module inHub-based embodiments of the present invention.

As shown in FIG. 4A, the Distribution Module is implemented by thefollowing components: (i) the Distribution Management Module, whichaddresses the streams of graphics commands and data to the differentGPPLs via chipset outputs, according to needs of the parallelizationschemes instantiated by the MMPGRS; (iii) Profiler module used ingraphics hub type system architectures, as illustrated in FIGS. 7B1through 7B12, so as to provide an additional source of Performance Data(i.e. beyond the GPUs, vendor's driver, and chipset); and (iv) a HubControl module, operating under control of the Distributed GraphicsFunction Control Module 409 within the AMCM 400, in graphics hub typesystem architectures, as illustrated in FIGS. 7B1 through 7B12, forconfiguring the Interconnect Network 404 according to the variousparallelization modes and to coordinate the overall functioning ofhardware components within the Recomposition Module across the graphicshub device (GHD) of the present invention.

As shown in FIG. 4A, the Distribution Module 402″ comprises threefunctional units: the Distribution Management, the Profiler, and the HubControl modules. The Distribution Management activates the Interconnectnetwork 404 to transfer command and data stream to the GPPLs. TheInterconnect network serves to (i) transfer command and data stream fromthe CPU to GPPLs, (ii) transfer raster data from GPPLs to theRecomposition Module, (iii) transfer raster data among GPPUs for analternative GPPL-based Recomposition, and (iv) conduct othercommunication tasks, such as profiling data, control, etc. among thevarious system components.

An exemplary embodiment of Interconnect for a cluster of 4 GPPLs is aconfigurable switch with 5 way PCI express x16 lanes, having oneupstream path between Hub and CPU, and 4 downstream paths between Huband four GPUs. It receives upstream of commands and data from the CPU,and transfers them downstream to GPPLs, under the control ofDistribution Management unit (of Distribution module), following thedata division scheme generated by the Division Control block ofDecomposition sub-module (2), according to ongoing parallel divisionmode. The switch can be set into one of the following possible transfersub-states: Divide, Broadcast, and Single. The Divide sub-state is setwhen the MMGPRS is operating in its Object Division Mode. The Broadcastsub-state is set when the MMGPRS is operating in its Image DivisionMode. The Single sub-state is set when the MMGPRS is operating in itsTime Division Mode, or in Single mode.

Within the Distribution Module, the Profiler Unit 407′ has severalfunctions in system architectures employing graphics hub device (GHD)sas illustrated in FIG. 7B 1 through 7B12, namely: (i) to deliver toDivision Control its own generated profiling data; (ii) to forward theprofiling data from GPUs to Division Control, due the fact that the GPUsare not directly connected to the host computing system in graphics hubbased system architectures, whereas they are in system architecturesillustrated in FIGS. 7A2 through 7A7-3; and (iii) to forward the Hubpre-GPU profiling data to the Division Control block within theDecomposition Module. Being close to the raw data passing by the GPUs,the Profiler 403 monitors the stream of geometric data and commands, forgraphics hub profiling purposes. Such monitoring operations involvepolygon, command, and texture count and quantifying data structures andtheir volumes for load balance purposes. The collected data is mainlyrelated to the performance of the geometry subsystem employed in eachGPU.

Within the Distribution Module of system architectures employing thegraphics hub device (GHD) of the present invention, illustrated in FIGS.7B1 through 7B12, the Hub Controller Module 409′, operates under controlof the Distributed Graphics Function Control Module 409 within theAutomatic Mode Control Module 400. The primary function performed bythis Hub Controller Module 409′ is to configure the Interconnect Network404 according to the various parallelization modes and to coordinate theoverall functioning of hardware components across the DistributionModule of the graphics hub device (GHD) of the present invention.

The Description of the Recomposition Module of the Present Invention

The primary function of the Recomposition Module 403 is to mergetogether, the partial results of multiple graphics pipelines, accordingto parallelization mode that is operative at any instant in time. Theresulting or final Frame Buffer (FB) is sent to the display device (viaprimary GPU, or directly Under the AMCM 400, the Recomposition Modulecan be set to three different decomposing sub-states (C=1 through C=3),according to FIG. 4C, namely: The Test Based Sub-state C=1; the ScreenBased Sub-state C=2; and the None State C=3. The Test based sub-statecarries out re-composition based on a test performed on partial framebuffer pixels. Typically, these tests include the depth test, thestencil test, or combination thereof. The Screen based sub-statecombines together parts of the final frame buffers, in a puzzle likefashion, creating a single image. The None sub-state, or submode, makesno merges, just moves one of the pipeline frame buffers to the display,as required in time division parallelism or in single GPU (Non-Parallel)mode of operation.

The Test-Based Compositing suits compositing during the Object DivisionMode. According to this method, sets of Z-buffer, stencil-buffer andcolor-buffer are read back from the GPU FBs to host's memory forcompositing. The pixels of color-buffers from different GPUs are mergedinto single color-buffer, based on per pixel comparison of depth and/orstencil values (e.g. at given x-y position only the pixel associatedwith the lowest z value is let out to the output color-buffer). This isa software technique to perform hidden surface elimination amongmultiple frame buffers required for the Object Division Mode. Framebuffers are merged based on depth and stencil tests. Stencil tests, withor without combination with depth test, are used in different multi-passalgorithms. The final color-buffer is down-loaded to the primary GPU fordisplay.

Screen-Based Compositing suits compositing during the Image DivisionMode

The Screen-Based compositing involves a puzzle-like merging of imageportions from all GPUs into a single image at the primary GPU, which isthen sent out to the display. This method is a much simpler procedurethan the Test-Based Compositing Method, as no tests are needed. Whilethe primary GPU is sending its color-buffer segment to display, theMerger Module reads back other GPUs color-buffer segments to host'smemory, for downloading them into primary GPU's FB for display.

The None Sub-state is a non-compositing option which involves moving theincoming Frame Buffer to the display. This option is used when nocompositing is required. In the Time Division Mode, a singlecolor-buffer is read back from a GPU to host's memory and downloaded toprimary GPU for display. In the Non-Parallel Mode (e.g. employing asingle GPPL), usually the primary GPPL is employed for rendering, sothat no host memory transit is needed.

In the illustrative embodiments, the Recomposition Module is realized byseveral modules: (i) the Merge Management Module which handles thereading of frame buffers and the compositing during the Test-Based,Screen-Based And None Sub-States; (ii) the Merger Module which is analgorithmic module that performs the different compositing algorithms,namely, Test Based Compositing during the Test-Based Sub-state, andScreen Based Compositing during the Screen-Based Sub-state; (iii)Profiler module used in graphics hub type system architectures, asillustrated in FIGS. 7B1 through 7B12, so as to provide an additionalsource of Performance Data (i.e. beyond the GPUs, vendor's driver, andchipset); and (iv) a Hub Control module, operating under control of theDistributed Graphics Function Control Module 409 within the AMCM 400, ingraphics hub type system architectures, as illustrated in FIGS. 7B1through 7B12, for configuring the Interconnect Network 404 according tothe various parallelization modes and to coordinate the overallfunctioning of hardware components within the Recomposition Moduleacross the graphics hub device (GHD) of the present invention.

In the case where the Interconnect needs to interconnect a cluster of 4GPPLs, a configurable switch can be used having 5 way PCI express x16lanes, with one upstream path between the Hub and CPU, and 4 downstreampaths between Hub and four GPUs. Under the local control of the MergeManagement and the Hub Controller in the Recomposition Module, theInterconnect (e.g. switch) also performs the following functions: (i)transferring read-back FB raster data from GPPLs to the Merger block ofRecomposition Module and returning composited image to primary GPPL, allunder orchestration of the Merge Management block; (ii) transferring theread-back FB raster data among GPPLs for GPPL-based recomposition, sothat the finally recomposited pixel data in final image is composited inthe primary GPPL; (iii) transferring additional data, e.g. profilerdata, to Decomposition module; and (iv) transferring control commandsacross the MMPGRS system.

Within the Recomposition Module, the Profiler unit 403 has severalfunctions in system architectures employing graphics hub device (GHD)sas illustrated in FIG. 7B 1 through 7B12, namely: (i) to deliver toDivision Control its own generated profiling data; (ii) to forward theprofiling data from GPUs to Division Control, due the fact that the GPUsare not directly connected to the host computing system in graphics hubbased system architectures, whereas they are in system architecturesillustrated in FIGS. 7A2 through 7A7-3; and (iii) to forward the Hubpost-GPU profiling data to the Division Control block within theDecomposition-Module. Being close to the raw data passing by the GPUs,the Profiler 403 monitors the stream of geometric data and commands, forgraphics hub profiling purposes. Such monitoring operations involvepolygon, command, and texture count and quantifying data structures andtheir volumes for load balance purposes. The collected data is mainlyrelated to the performance of the geometry subsystem employed in eachGPU.

Another function performed by the Profiler 403 within the RecompositionModule is to profile the merge process and monitor the task completionof each GPU for load balancing purposes. In graphics hub device (GHD)class of system architecture illustrated in FIGS. 7B1 through 7B12, bothProfilers 407′ and 403′, in the Distribution and Recomposition Modules,unify their collected Performance Data and deliver the unifiedperformance data, as feedback, to the Automatic Mode Control Module(AMCM) via the Decomposition Module, as shown in FIG. 4A. Notably, thecommunication linkage between the two Profiling blocks is achieved usingthe Interconnect network 404. In some illustrative embodiments, the two“pre-GPU” and “post-GPU” units of the graphics hub device (GHD), formedby the components within the Distribution and Recomposition Modules ofsystem architectures illustrated in FIGS. 7B1 through 7B12, may resideon the same silicon chip, having many internal interconnections, whereasin other illustrative embodiments, these subcomponents may be realizedon different pieces of silicon or functionally like semiconductormaterial, used to fabricate the graphics hub device (GHD)s of thepresent invention within diverse embodiments of the MMPGRS of thepresent invention.

Within the Recomposition Module of system architectures employing thegraphics hub device (GHD) of the present invention, illustrated in FIGS.7B1 through 7B12, the Hub Controller Module 409′, operates under controlof the Distributed Graphics Function Control Module 409 within the AMCM400. The primary function performed by this Hub Controller Module 409′is to configure the Interconnect Network 404 according to the variousparallelization modes and to coordinate the overall functioning ofhardware components across the Recomposition Module of the graphics hubdevice (GHD) of the present invention.

Notably, in some illustrative embodiments of the present invention, theHub Controllers 409′, in the Distribution and Recomposition Modules insystem architectures embraced by the graphics hub device (GHD) of thepresent invention, can be realized as a single device or unit, on thesame piece of silicon or like semiconductor material. In otherembodiments, the Hub Controllers 409′ can be realized as discrete units,on the same piece of silicon or like semiconductor material, or onseparate pieces of silicon material (e.g. on different chips sets).

Description of the Automatic Mode Control Module (AMCM) 400 Within theMMPGRS of the Present Invention

During the run-time of any graphics-based application on the hostcomputing system, the MMPGRS renders and displays the graphicsenvironment being generated by the application, which typically willinclude many dynamically changing scenes, where the plot unfolds, andeach scene typically involves a sequence of many image frames. Suchscenes could involve virtually anything, including a forest with manyleaves moving in the wind, a lake with many reflections, or a closedspace in a castle with many light sources. Such scenes require parallelrendering and the role of the MMPGS is to automatically determine whichmode of parallel operation will result in optimal performance on thehost computing system.

As shown in FIG. 4A, the Automatic Mode Control Module (AMCM) 400comprises three algorithmic modules, namely: an Application Profilingand Analysis Module 407; a Parallel Policy Management Module 408; and aDistributed Graphics Function Control.

In the preferred embodiment shown in FIG. 4B, the AMCM also comprisestwo data stores: a Historical Repository 404; and a Application/SceneProfile Database 405. The primary function of the AMCM is to control thestate of Multi-mode Parallel Rendering Subsystem 410 by virtue of itsflexible multi-state behavior and fast interstate transitioncapabilities.

As shown in FIG. 4C, the AMCM 400 comprises a User Interaction Detection(UID) Subsystem 438 which includes a Detection and Counting Module 433in combination with a UID Transition Decision Module 436. Thesesubsystems and modules will be described in greater detail hereinbelow.

Overview on The Automatic Mode Control Module (AMCM) in the MMPGRS ofthe Present Invention

When implementing the Automatic Mode Control Module or Mechanism (AMCM)in the MMPGRS of the present invention, there are several classes oftechniques which can be usefully applied, to determine when and how toswitch into, out of, and transition between modes of parallel operationon the MMPGRS platform, during the run-time of a particulargraphics-based application, and optimize system performance, namely:Mode Control Techniques Based On Scene/Application Profiling; and ModeControl Techniques Based On System-User Interaction Detection. It isappropriate at this juncture to provide an overview on the varioustechniques that the AMCM can use to best determine how to automaticallycontrol the mode of parallel operation on the MMPGRS Platform, andoptimize system performance.

Description Of Mode Control Techniques Employed Within The AMCM Based OnScene/Application Profiling

(1) Real-Time Profiling of Scenes (on a Frame-by-Frame Basis):

This method involves collecting and analysis of Performance Data, duringapplication run-time, in order to construct scene profiles for imageframes associated with particular scenes in a particular graphics-basedapplication, and maintaining these scene profiles in theScene/Application Profile Database within the AMCM. This way, duringrun-time, the AMCM can access and use these scene profiles so as to bestdetermine how to dynamically control the modes of parallel operation ofthe MMPGRS to optimize system performance. As will be described ingreater detail hereinafter, this technique can be practiced using theApplication Profiling and Analysis Module 407 and Parallel PolicyManagement Module 408 illustrated in FIGS. 4A, 4D, 5C1, 5C2, 5C3, and5C4, in the context of highly diverse MMPGRS system architectures, aswell as within multi-user application environments supported overdistributed network environments, as shown in FIGS. 12A and 12B.

(2) Real-Time Detection of Scene Profile indices Directly ProgrammedWithin Pre-Profiled Scenes of Particular Applications:

This technique involves analyzing, prior to run-time, the scenes of aparticular application, and then indexing the scene with Scene ProfileIndices and storing corresponding Mode Control Parameters (MCPs) (e.g.Switch to Object Division Mode) within the local Scene/ApplicationProfile Database within the AMCM, or other data storage device that isaccessible in real-time by the AMCM during application run-time. Thenduring run-time, the AMCM automatically detects the scene, and consultsthe Scene Profile Indices for the corresponding MCPs from theScene/Application Profile Database so as to best determine how todynamically control the modes of parallel operation of the MMPGRS tooptimize system performance.

(3) Real-Time Detection of Mode Control Command (MCCs) DirectlyProgrammed within Pre-Profiled Scenes of Particular Applications:

This technique involves, prior to run-time (e.g. during game applicationdevelopment), analyzing the scenes of a particular application, and thedirectly programming Mode Control Commands (MMCs) (e.g. Switch to ObjectDivision Mode) within the individual image frames of each scene,following standards to be established and followed by developers in thecomputer graphics industry. Then during run-time, the MCM automaticallydetects these MCCs along the graphics command and data stream, and usethese commands so as to best determine how to dynamically control themodes of parallel operation of the MMPGRS to optimize systemperformance.

Description of Mode Control Techniques Employed within the AMCM Based onSystem-User Interaction Detection

This approach, which can be used in conjunction with any of the aboveScene/Application Profiling Techniques, involves automatically detectingthe users interaction with the host computing system (e.g. mouse devicemovement, keyboard depressions, etc) and providing this Interaction Datato the AMCM so that it can best determine how to dynamically control themodes of parallel operation of the MMPGRS to optimize systemperformance, given the user's interaction with the host computingsystem, or application running thereon, at any instance in time. As willbe described in greater detail hereinafter, this technique can bepracticed using the UID Subsystem 438 illustrated in FIGS. 5A, 5B and5C2.

The Application Profiling and Analysis Module

As shown in FIG. 4D, the Application Profiling and Analysis Module 407monitors and analyzes Performance and Interactive data streamscontinuously acquired by profiling the Application while its running. InFIG. 5D, the Performance Data inputs provided to the ApplicationProfiling and Analysis Module include: texture count; screen resolution;polygon count; utilization of geometry engine, pixel engine, videomemory and GPPL; the total pixels rendered, the total geometric datarendered; the workload of each GPPL; the volumes of transferred data.The System-User Interactive (Device) Data inputs provided to theApplication Profiling and Analysis Module include: mouse movement; headmovement; voice commands; eye movement; feet movement; keyboard; LAN,WAN or Internet (WWW) originated application (e.g. game) updates.

The Tasks performed by the Application Profiling and Analysis Moduleinclude: Recognition of the Application; Processing of Trial and ErrorResults; Utilization of Application Profile from Application/SceneProfile Database; Data Aggregation in the Historical Depository;Analysis of input performance data (frame-based); Analysis based onintegration of frame-based “atomic” performance data, aggregated data atHistorical Depository, and Application/Scene Profile Database data;Detection of rendering algorithms used by Application; Detection of useof FB in next successive frame; Recognition of preventative conditions(to parallel modes); Evaluation of pixel layer depth; Frame/secondcount; Detection of critical events (e.g. frames/sec/drop); Detection ofbottlenecks in graphics pipeline; Measure of load balance among GPUs;Update Application/Scene Profile Database from the HistoricalDepository; and Recommendation on optimal parallel scheme.

The Application Profiling and Analysis Module performs its analysisbased on the following:

(1) The Performance Data collected from several sources, such asvendor's driver, GPUs, chipset, and optionally—from the graphics Hubembodiments of the present invention, described in greater detailhereinafter;

(2) Historical repository 404 which continuously stores up the acquireddata (i.e. this data having historical depth, and being used forconstructing behavioral profile of ongoing application); and

(3) Knowledge based Application/Scene Profile Database 405 which is anapplication profile library of prior known graphics applications (andfurther enriched by newly created profiles based on data from theHistorical Depository).

In the MMGPRS of the illustrative embodiment, the choice of parallelrendering mode at any instant in time involves profiling and analyzingthe system's performance by way of processing both Performance DataInputs and Interactive Device Inputs, which are typically generated froma several different sources within MMPGRS, namely: the GPUs, thevendor's driver, the chipset, and the graphic Hub (optional).

Performance Data needed for estimating system performance and locatingcasual bottlenecks, includes:

-   -   (I) Texture Count;    -   (Ii) Screen Resolution;    -   (Iii) Polygon Volume;    -   (iv) at each GPPL, utilization of    -   (a) the Geometry Engine    -   (b) the Pixel engine, and    -   (c) Video memory;    -   (v) Utilization of the CPU;    -   (vi) total pixels rendered;    -   (vii) total geometric data rendered;    -   (viii) workload of each GPU; and    -   (ix) volumes of transferred data.

As shown in FIG. 4D, this Performance Data is fed as input into theApplication Profiling and Analysis Module for real-time processing andanalysis Application Profiling and Analysis Module. In the illustrativeembodiment, the Application Profiling and Analysis Module performs thefollowing tasks:

-   -   (1) Recognition of Application (e.g. video game, simulation,        etc.);    -   (2) Processing of trial & error results produced by the        processes described in FIGS. 5C3 and 5C4;    -   (3) Utilization of the Application Profile from data in the        Application/Scene Profile Database;    -   (4) Aggregation of Data in the Historical Repository;    -   (5) Analysis of Performance Data Inputs;    -   (6) Analysis based on the integration of    -   (a) Frame-based “atomic” Performance Data,    -   (b) Aggregated data within the Historical Repository, and    -   (c) Data stored in the Application/Scene Profile Database;    -   (7) Detection of rendering algorithms used by Application    -   (8) Detection of use of the FB in next successive frame as a        preventive condition for Time Division Mode;    -   (9) Recognition of preventive conditions for other parallel        modes;    -   (10) Evaluation of pixel layer depth at the pixel subsystem of        GPU;    -   (11) Frame/sec count;    -   (12) Detection of critical events (e.g. frame/sec drop);    -   (13) Detection of bottlenecks in graphics pipeline;    -   (14) Measure and balance of load among the GPUs;    -   (15) Update Application/Scene Profile Database from data in the        Historical Depository; and    -   (16) Selection of the optimal parallel graphics rendering mode        of operation for the MMPGRS.        The Parallel Policy Management Module

Parallel Policy Management Module 408 makes the final decision regardingthe preferred mode of parallel graphics rendering used at any instant intime within the MMPGRS, and this decision is based on the profiling andanalysis results generated by the Application Profiling and AnalysisModule. The decision is made on the basis of some number N of graphicsframes. As shown above, the layer depth factor, differentiating betweenthe effectiveness of the Object Division vs. Image Division Mode, can beevaluated by analyzing the relationship of geometric data vs. fragmentdata at a scene, or alternatively can be found heuristically.Illustrative control policies have been described above and in FIGS. 5C1through 5C3.

The Distributed Graphic Function Control Module

Distributed Graphic Function Control Module 409 carries out all thefunctions associated with the different parallelization modes, accordingto the decision made by the Parallel Policy Management Module. TheDistributed Graphic Function Control Module 409 drives directly theconfiguration sub-states of the Decomposition, Distribution andRecomposition Modules, according to the parallelization mode. Moreover,Application Profiling and Analysis includes drivers needed for hardwarecomponents such as graphic Hub, described hereinafter in the presentPatent Specification.

State Transitions within the MMPGRS of the Illustrative Embodiment ofthe Present Invention

As shown in the state transition diagram of FIG. 6A, the MMPGRS of theillustrative embodiment has six (6) system states. Three of these systemstates are parallel graphics rendering states, namely: the ImageDivision State, which is attained when the MMPGRS is operating in itsImage Division Mode; the Object Division State, which is attained whenthe MMPGRS is operating in its Object Division Mode; and the TimeDivision State, which is attained when the MMPGRS is operating in itsTime Division Mode. The system also includes a Non-Parallel GraphicsRendering State, which is attained only when a single GPPL isoperational during the graphics rendering process. There is also anApplication Identification State, and a Trial & Error Cycle State. Asillustrated in FIG. 4C and FIG. 6A, each parallelization state ischaracterized by sub-state parameters A, B, C. As shown in the statetransition diagram of FIG. 6A, the Non-Parallel (i.e. Single GPPL) Stateis reachable from any other state of system operation.

In accordance with the principles of the present invention, profiles ofall previously analyzed and known graphics-based Applications are storedin the Application/Scene Profile Database 405 of the MMPGRS. Wheneverthe graphics-based application starts, the system enters ApplicationIdentification State, and the AMCM attempts to automatically identifywhether this application is previously known to the system. In the caseof a previously known application, the optimal starting state isrecommended by the Database, and the system transitions to that systemstate. Further on, during the course of the application, the AMCM isassisted by the Application/Scene Profile Database to optimize theinter-state tracking process within the MMPGRS. In the case of anapplication previously unknown to the MMPGRS, the Trial & Error CycleState is entered, and attempts to run all three parallelization schemes(i.e. Modes) are made for a limited number of cycles.

During the course of the Application, the decision by the system as towhich mode of graphics rendering parallelization to employ (at anyinstant in time) is supported either by continuous profiling andanalysis, and/or by trial and error. The Trial and Error Process isbased on comparing the results of a single, or very few cycles spent bythe system at each parallelization state.

During the course of continuous profiling and analysis by theApplication Profiling and Analysis Module 407, the following parametersare considered and analyzed by the AMCM with respect to each state/modetransition decision:

-   -   Pixel processing load    -   Screen resolution    -   Depth complexity of the scene    -   Polygon count    -   Video-memory usage    -   Frame/second rate    -   Change of frames/second rate    -   Tolerance of latency    -   Use of the same FB in successive frame    -   (10) User-System Interaction during the running of the        Application.        User-Interactivity Driven Mode Selection Within The MMPGRS of        the Present Invention

Purely in terms of “frames/second” rate, the Time Division Mode is thefastest among the parallel graphics rendering modes of the MMGRS, andthis is by virtue of the fact that the Time Division Mode worksfavorably to reduce geometry and fragment bottlenecks by allowing moretime. However, the Time Division Mode (i.e. method) of parallelizationdoes not solve video memory bottlenecks. Also, the Time Division Modesuffers from other problems, namely: (i) CPU bottlenecks; (ii) theunavailability of GPU-generated frame buffers to each other, in caseswhere the previous frame is required as a start point for the successiveframe; and also (iii) from pipeline latency. Automatic transition of theMMGPRS to its Object-Division Mode effectively releases the system fromtransform and video memory loads. In many applications, these problemsprovide reasons not for the MMPGS to use or enter into its Time DivisionMode. However, for some other applications, the Time Division Mode maybe suitable and perform better than other parallelization schemesavailable on the MMGPRS of the present invention (e.g. Object-DivisionMode and Image-Division Mode).

During the Time Division Mode, the pipeline latency problem arises onlywhen user-system interaction occurs. Also, in many interactive gamingapplications (e.g. video games), often there are 3D scenes withintervals of user-system interactivity during the Time Division Mode.Thus, in order to achieve the highest performance mode of parallelgraphics rendering at runtime, the MMPGRS of the present inventionemploys a User Interaction Detection (UID) Subsystem 438 which enablesautomatic and dynamic detection of the user's interaction with thesystem. Absent preventive conditions (such as CPU bottlenecks and needfor the same FB in successive frames), the UID subsystem 438 enablestimely automated implementation of the Time Division Mode only when nouser-system interactivity is detected so that system performance isautomatically optimized.

These and other constraints are taken into account during theinter-modal transition process, as illustrated in the State TransitionDiagram of FIG. 6A, and described below:

Transition from Object Division to Image Division follows a combinationof one or more of the following conditions:

-   -   Increase in pixel processing load    -   Increase in screen resolution    -   Increase in scene depth complexity    -   Decrease in polygon count

Transition from Image Division to Object Division follows a combinationof one or more of the following conditions:

-   -   Increase of polygon count    -   Increase of video memory footprint    -   Decrease of scene depth complexity

Transition from Object Division to Time Division follows a combinationof one or more of the following conditions:

-   -   Demand for higher frame/second rate    -   Higher latency is tolerated    -   There is no use of the FB for successive frame    -   No predefined input activity detected by the UID Subsystem

Transition from Time Division to Object Division follows a combinationof one or more of the following conditions:

-   -   Latency is not tolerable    -   FB is used for successive frame    -   High polygon count    -   Input activity detected by the UID Subsystem

Transition from Time Division to Image Division follows a combination ofone or more of the following conditions:

-   -   Latency is not tolerable    -   FB is used for successive frame    -   High pixel processing load    -   Input activity detected by the UID Subsystem

Transition from Image Division to Time Division follows a combination ofone or more of the following conditions:

-   -   Demand for higher frame/second rate    -   Latency is tolerable    -   High polygon count    -   No predefined input activity detected by the UID Subsystem.

In the illustrative embodiment, this capacity of the MMPGRS is realizedby the User Interaction Detection (UID) Subsystem 438 provided withinthe Application Profiling and Analysis Module 407 in the Automatic ModeControl Module of the system. As shown in FIG. 5A, the UID subsystem 438comprises: a Detection and Counting Module 433 in combination with a UIDTransition Decision Module 436.

As shown in FIGS. 5A and 4D, the set of interactive devices which cansupply User Interactive Data to the UID subsystem can include, forexample, a computer mouse, a keyboard, eye-movement trackers,head-movement trackers, feet-movement trackers, voice commandsubsystems, Internet, LAN, WAN and/or Internet originateduser-interaction or game updates, and any other means of userinteraction detection, and the like.

As shown in FIG. 5A, each interactive device input 432 supported by thecomputing system employing the MMPGRS feeds User Interaction Data to theDetection and Counting Module 433 which automatically counts the elapsedpassage of time for the required non-interactive interval. When such atime interval is counted or has elapsed (i.e. without detection ofuser-system interactivity), the Detection and Counting Module 433automatically generates a signal indicative of this non-interactivity(434) which is transmitted to the UID Transition Decision Module 436.Thereafter, UID Transition Decision Module 436 issues a state transitioncommand (i.e. signal) to the Parallel Policy Management Module 408,thereby causing the MMPGRS to automatically switch from its currentlyrunning parallel mode of graphics rendering operation, to its TimeDivision Mode of operation. During the newly initiated Time DivisionMode, whenever system-user interactivity from the interactive device isdetected 432 by the Detection and Counting Module 433, an system-userinteractivity signal 435 is transferred to the UID Transition DecisionModule 436, thereby initiating the system to return from the thencurrently Time Division Mode, to its original parallel mode of operation(i.e. the Image or Object Division Mode, as the case may be).

As shown in FIG. 5A, an Initialization Signal 431 is provided to theDetection and Counting Module 433 when no preventive conditions for TimeDivision exist. The function of the Initialization Signal 431 is to (1)define the set of input (interactive) devices supplying interactiveinputs, as well as (2) define the minimum elapsed time period with nointeractive activity required for transition to the Time Division Mode(termed non-interactive interval). The function of the UID TransitionDecision Module 436 is to receive detected inputs 435 and no inputs 434during the required interval, and, produce and provide as output, asignal to the Parallel Policy Management System, initiating a transitionto or from the Time Division Mode of system operation, as shown.

In applications dominated by Image Division or Object Division Modes ofoperation, with intervals of non-interactivity, the UID Subsystem 438within the MMGPRS can automatically initiate a transition into its TimeDivision Mode upon detection of user-interactivity, without the systemexperiencing user lag. Then as soon as the user is interacting with theapplication, the UID Subsystem of the MMGPRS can automaticallytransition (i.e. switch) the system back into its dominating mode (i.e.the Image Division or Object Division). The benefits of this method ofautomatic “user-interaction detection (UID)” driven mode controlembodied within the MMGRPS of the present invention are numerous,including: best performance; no user-lag; and ease of implementation.

Notably, the automated event detection functions described above can beperformed using any of the following techniques: (i) detecting whetheror not a mouse movement or keyboard depression has occurred within aparticular time interval (i.e. a strong criterion); (ii) detectingwhether or not the application (i.e. game) is checking for such events(i.e. a more subtle criterion); or (iii) allowing the application's gameengine itself to directly generate a signal indicating that it isentering an interactive mode.

The state transition process between Object-Division/Image-DivisionModes and the Time Division Mode initiated by the UID subsystem of thepresent invention is described in the flow-chart shown in FIG. 5B. Asshown therein, at Block A, the UID subsystem is initialized. At Block B,the time counter of the Detection and Counting Module 433 isinitialized. At Block C, the UID subsystem counts for the predefinednon-interactive interval, and the result is repeatedly tested at BlockD. When the test is positively passed, the parallel mode is switched tothe Time-Division at Block E by the Parallel Policy Management Module.At Block F, the UID subsystem determines whether user interactive input(interactivity) has been detected, and when interactive input has beendetected, the UID subsystem automatically returns the MMPGRS to itsoriginal Image or Object Division Mode of operation, at Block G of FIG.5B.

As will be described in greater detail below, the entire process ofUser-Interactivity-Driven Mode Selection occurs within the MMPGRS of thepresent invention when N successive frames according control policy arerun in either the Object Division or Image Division Mode of operation,as shown during Blocks I and J of FIGS. 5C1 and 5C2.

Operation of the Automatic Mode Control Cycle within The MMPGRS of thePresent Invention

Referring to FIG. 5C 1, the Profiling and Control Cycle Process withinthe MMPGRS will now be described in detail, wherein each statetransition is based on above listed parameters (i.e. events orconditions) (1) through (6) listed above, and the UID Subsystem isdisabled. In this process, Steps A through C test whether the graphicsapplication is listed in the Application/Scene Profile Database of theMMPGRS. If the application is listed in the Application/Scene ProfileDatabase, then the application's profile is taken from the Database atStep E, and a preferred state is set at Step G. During Steps I-J, Nsuccessive frames are rendered according to Control Policy, under thecontrol of the AMCM with its UID Subsystem disabled. At Step K,Performance Data is collected, and at Step M, the collected PerformanceData is added to the Historical Repository, and then analyzed for nextoptimal parallel graphics rendering state at Step F. Upon conclusion ofapplication, at Step L, the Application/Scene Profile Database isupdated at Step N using Performance Data collected from the HistoricalRepository.

Referring to FIG. 5C 2, the Profiling and Control Cycle Process withinthe MMPGRS will now be described in detail, with the UID Subsystem isenabled. In this process, Steps A through C test whether the graphicsapplication is listed in the Application/Scene Profile Database of theMMPGRS. If the application is listed in the Application/Scene ProfileDatabase, then the application's profile is taken from the Database atStep E, and a preferred state is set at Step G. During Steps I-J, Nsuccessive frames are rendered according to Control Policy under thecontrol of the AMCM with its UID Subsystem enabled and playing an activerole in Parallel Graphics Rendering State transition within the MMPGRS.At Step K, Performance Data is collected, and at Step M, the collectedPerformance Data is added to the Historical Repository, and thenanalyzed for next optimal parallel graphics rendering state at Step F.Upon conclusion of application, at Step L, the Application/Scene ProfileDatabase is updated at Step N using Performance Data collected from theHistorical Repository.

Operation of the Periodical Trial & Error Process of the PresentInvention within the MMPGRS of the Present Invention

As depicted in FIG. 5C 3, the Periodical Trial & Error Process differsfrom the Profiling and Control Cycle Process/Method described above,based on its empirical approach. According the Periodical Trial & ErrorProcess, the best parallelization scheme for the graphical applicationat hand is chosen by a series of trials described at Steps A through Min FIG. 5C 3. After N successive frames of graphic data and commands areprocessed (i.e. graphically rendered) during Steps N through O, anotherperiodical trial is performed at Steps A through M. In order to omitslow and not necessary trials, a preventive condition for any ofparallelization schemes can be set and tested during Steps B, E, and H,such as used by the application of the Frame Buffer FB for the nextsuccessive frame, which prevents entering the Time Division Mode of theMMPGRS.

In the flowchart of FIG. 5C 4, a slightly different Periodical Trial &Error Process (also based on an empirical approach) is disclosed,wherein the tests for change of parallel graphics rendering state (i.e.mode) are done only in response to, or upon the occurrence of a drop inthe frame-rate-per-second (FPS), as indicated during Steps O, and Bthrough M.

Conditions For Transition Between Object and Image Division Modes ofOperation in the MMPGRS of the Present Invention

In a well-defined case, Object Division Mode supersedes the ImageDivision Mode in that it reduces more bottlenecks. In contrast to theImage Division Mode that reduces only the fragment/fill bound processingat each GPU, the Object Division Mode relaxes bottleneck across thepipeline: (i) the geometry (i.e. polygons, lines, dots, etc) transformprocessing is offloaded at each GPU, handling only 1/N of polygons(N—number of participating GPUs); (ii) fill bound processing is reducedsince less polygons are feeding the rasterizer: (iii) less geometrymemory is needed; and (iv) less texture memory is needed.

Automated transition to the Object Division State of operationeffectively releases the MMPGRS of the present invention from transformand video memory loads. However, for fill loads, the Object DivisionState of operation will be less effective than the Image Division Stateof operation.

At this juncture it will be helpful to consider under what conditions atransition from the Object Division State to the Image Division Statecan occur, so that the parallel graphics system of the present inventionwill perform better “fill loads”, especially in higher resolution.

Notably, the duration of transform and fill phases differ between theObject and Image Division Modes (i.e. States) of operation. For claritypurposes, consider the case of a dual-GPU graphics rendering system.Rendering time in the Image Division Mode is given by:T _(ObjDiv)=Transform+Fill/2  (1)whereas in Object Division Mode, the fill load does not reduce in thesame factor as transform load.The render time is:T_(ImgDiv)=Transform/2+DepthComplexity*Fill/2  (2)The fill function Depth Complexity in Object Division Mode depends ondepth complexity of the scene. Depth complexity is the number offragment replacements as a result of depth tests (the number of polygonsdrawn on every pixel). In the ideal case of no fragment replacement(e.g. all polygons of the scene are located on the same depth level),the second component of the Object Division Mode reduces to:T _(ImgDiv)=Transform/2+Fill/2  (2.1)However, when depth complexity becomes high, the advantage of the ObjectDivision Mode drops significantly, and in some cases the Image DivisionMode may even perform better (e.g. in Applications with small number ofpolygons and high volume of textures).The function DepthComplexity denotes the way the fill time is affectedby depth complexity: $\begin{matrix}{{DepthComplexity} = \frac{2{E\left( {L/2} \right)}}{E(L)}} & (3)\end{matrix}$where E(L) is the expected number of fragments drawn at pixel for Ltotal polygon layers. In ideal case DepthComplexity=1. In this case, Eis given by: $\begin{matrix}{{E(m)} = {1 + {\frac{1}{m}\left( {\sum\limits_{i = 1}^{m - 1}{E(i)}} \right)}}} & (3.1)\end{matrix}$For a uniform layer-depth of L throughout the scene, the followingalgorithm is used to find conditions for switching from the ObjectDivision Mode to the Image Division Mode: $\begin{matrix}{{{chose\_ div}{\_ mode}\left( {{Transform},{Fill}} \right)} = \left\{ \begin{matrix}{ObjectDivision} & {{{Transform} + \frac{Fill}{2}} > {\frac{Transform}{2} + {\frac{Fill}{2} \times {DepthComplexity}}}} \\{ImageDivision} & {otherwise}\end{matrix} \right.} & (4)\end{matrix}$In order to choose between the Image Division and the Object DivisionMode, an algorithm is used which detects which transform and fill boundprocessing is smaller. Once the layer-depth reaches some threshold valuethroughout the scene, the Object Division Mode will not minimize theFill function any more.

Example: Consideration of A General Scene:Denote the time for drawing n polygons and p pixels as Render(n,p), andallow P to be equal to the time taken to draw one pixel. Here thedrawing time is assumed to be constant for all pixels (which may be agood approximation, but is not perfectly accurate). Also, it is assumedthat the Render function, which is linearly dependent on p (the numberof pixels actually drawn), is independent of the number of non-drawingsthat were calculated. This means that if the system has drawn a bigpolygon that covers the entire screen surface first, then for anyadditional n polygons: Render(n,p)=p×P. $\begin{matrix}{{{Render}\left( {n,p} \right)} = {\sum\limits_{i = 1}^{\infty}{P \times {\left\{ {\left. x \middle| {{LayerDepth}(x)} \right. = i} \right\} } \times {E(i)}}}} & (5)\end{matrix}$

The screen space of a general scene is divided into sub-spaces based onthe layer-depth of each pixel. This leads to some meaningful figures.

For example, suppose a game engine generates a scene, wherein most ofthe screen (90%) has a depth of four layers (the scenery) and a smallpart is covered by the player (10%) with a depth of 20 layers. WithoutObject Division Mode support, the value of Render function is given by:

Render(n,p)=p×0.9×E(4)+p×0.1×E(20)=2.2347739657143681×p

With Object Division Mode support, the value of the Render function is:Render(n/2,p)=p×0.9×E(4/2)+p×0.1×E(20/2)=1.6428968253968255×pNotably, in this case, the improvement factor when using Object DivisionMode support is 1.3602643398952217. On the other hand, a CAD enginemight have a constant layer depth of 4. The improvement factor forinteresting cases is shown in a table set forth in copending applicationSer. No. 11/789,039, supra.

It is easily seen from that table that when the layer depthDepthComplexity becomes larger, the Object Division Mode does notimprove the rendering time by a large amount, and if rendering time isthe bottleneck of the total frame calculation procedure, then the ImageDivision Mode might be a better approach. The analysis results by theApplication Profiling and Analysis Module are passed down to the nextmodule of Parallel Policy Management Module.

Parallel Graphics Rendering Process of the Present Invention PerformedDuring Each Mode of Parallelism on the MMPGRS

The parallel graphics rendering process performed during each mode ofparallelism on the MMPGRS will now be described with reference to theParallel Graphics Processing Pipeline Model of FIG. 6B and flowchartsset forth in FIGS. 6C1, 6C2 and 6C3, for the Image, Time and ObjectDivision Modes, respectively.

Parallel Graphics Rendering Process for a Single Frame During the ImageDivision Mode of the MMPRS of the Present Invention

In FIG. 6C 1, the parallel graphics rendering process for a single frameis described in connection with the Image Division Mode of the MMPRS ofthe present invention. In the Image Division Mode, the Decomposition,Distribution and Recomposition Modules are set as follows: theDecomposition Module is set on sub-state A-2, the Distribution Module isset on sub-state B-2, and the Recomposition Module is set on sub-stateC-2. The Decomposition Module splits up the image area into sub-imagesand prepares partition parameters for each GPPL 6120. Typically, thepartition ratio is dictated by the Automatic Mode Control Module basedon load balancing considerations. The physical distribution of theseparameters among multiple GPPLs is done by the Distribution Module(6124). From this point on the stream of commands and data (6121) isbroadcasted to all GPPLs for rendering (6123), unless end-of-frame isencountered (6122). When rendering of frame is accomplished, each GPPLholds a different part of the entire image. Compositing of these partsinto final image is done by the Recomposition Module moving all partialimages (i.e. color-FB) from secondary GPPLs to the primary GPPL (6125),merging the sub-images into final color-FB (6126), and displaying the FBon the display screen (6127).

Parallel Graphics Rendering Process for a Single Frame During the TimeDivision Mode of the MMPRS of the Present Invention

In FIG. 6C 2, the parallel graphics rendering process for a single frameis described in connection with the Time Division Mode of the MMPRS ofthe present invention. In the Time Division Mode, the Decomposition,Distribution and Recomposition Modules are set as follows: theDecomposition Module is set on sub-state A-3, the Distribution Module isset on sub-state B-3, and the Recomposition Module is set on sub-stateC-3. The Decomposition Module aligns a queue of GPPLs 6130, appoints thenext frame to the next available GPPL 6131, and monitors the stream ofcommands and data to all GPPLs 6132. The physical distribution of thatGCAD stream is performed by the Distribution Module 6134. Upon detectionof an end-of-frame command 6133 at one of the GPPLs, the control movesto the Recomposition Module which moves the color-FB of the completingsecondary GPPL, to the primary GPPLs 6135. The primary GPPL thendisplays the recomposited image in the FB on the display screen 6136 ofthe display device.

Parallel Graphics Rendering Process For a Single Frame During the ObjectDivision Mode of the MMPRS of the Present Invention

In FIG. 6C 3, the parallel graphics rendering process for a single frameis described in connection with the Object Division Mode of the MMPRSimplemented according to the software-based architecture of the presentinvention. In the Object Division Mode, the Decomposition, Distributionand Recomposition Modules are set as follows: the Decomposition Moduleis set on sub-state A-1, the Distribution Module is set on sub-stateB-1, and the Recomposition Module is set on sub-state C-1. TheDecomposition Module activity starts with interception of graphicscommands 6140 on their way between standard graphics library (e.g.OpenGL, Dirct3D) and vendor's GPU driver. Each graphics command istested for blocking mode 6142, 6143 and state operation class 6144.Blocking operations are exceptional in that they require a composedvalid FB data, thus in the Object Division Mode, they have an inter-GPPLeffect. Therefore, whenever one of the blocking operations is issued,all the GPPLs must be synchronized. Each frame has at least 2 blockingoperations: Flush and Swap, which terminate the frame. State operations(e.g. definition of light source) have an across the board effect on allGPPLS. In both cases the command must be duplicated to all GPPLs, ratherthan delivered to one of them. Therefore the Distribution Modulephysically sends the command to all GPPLs 6150. On the other hand, aregular command that passed the above tests is designated to a singletarget GPPL 6145, and sent by Distribution Module to that GPPL 6151.

When a blocking mode command is detected 6143, a blocking flag is set on6147 indicating blocking state. At this point, a composition of allframe buffers must occur and its result be duplicated to all GPPL. Therendering of upcoming commands is mirrored (duplicated) at all of theGPPL, unless an end-of-blocking mode is detected. The compositingsequence includes issuing of a flushing command 6149 to empty thepipeline. Such a command is sent to all GPPLs 6152. Then at each GPPLthe color and Z Frame Buffer are read back to host memory 6154, and allcolor Frame Buffers are composited based on Z and stencil buffers 6156.Finally, the resulting Frame Buffer is sent to all GPPLs 6160. Allsuccessive graphics commands will be duplicated (i.e. replicated) to allGPPLs generating identical rendering results, unless the blocking modeflag is turned off. When the end-of-blocking mode is detected 6146, theblocking flag is turned off 6148 and regular object division is resumed.

When detected 6144 by the Decomposition Module, state operation commands(e.g. glLight, glColor) are being duplicated to ail GPPLs 6150. UponEnd-of-frame detection 6141, a compositing process is taking place 6153,6155, 6157, 6158, very similar to that of blocking mode. However themerging result is sent to the display screen connected to the primaryGPPL.

Illustrative Designs for the Multi-Mode Parallel Graphics RenderingSystem (MMPGRS) of the Present Invention Having Diverse SystemArchitectures Parallelizing The Operation of Multiple GraphicsProcessing Pipelines (GPPLs)

FIG. 7A 1-1 sets forth a schematic diagram that illustrates differentenvironments for practicing the embodiments of MMPGRS present invention,namely: Host Memory Space (HMS), Processor/CPU Die Space, Bridge Circuit(IGD) Space, Graphics Hub Space, and External GPU Space.

In FIG. 7A 1-2, the table describes eleven (11) different Classes ofMMPGRS Architecture, defined in terms of the Architectural Spacesspecified in FIG. 7A 1-1 in which the primary MMPGRS components areembodied in any particular Class of MMPGRS Architecture, namely: HostMemory Space HMS (software); HMS+IGD; HMS+Fusion; HMS+Multicore;HMS+GPU-Recomposition; HUB; HUB+GPU-Recomposition; Chipset; CPU/GPUFusion; Multicore CPU; and Game Console.

The MMPGRS Architecture Table (i.e. Map) of the illustrative embodimentsof the present invention illustrates several things.

First, within each MMPGRS Architecture, illustrated in FIG. 7A 1-2, theAutomatic Mode Control Module (AMCM) 400 and Modules and Submodules ofthe Multimode Parallel Graphics Rendering Subsystem 401, 402, 403 mayreside in the different kinds of Architectural Space specified in FIG.7A 1-1, while multiple GPPLs, distributed in various ways in suchenvironment, are driven multiple modes of parallelism that aredynamically managed in accordance with the principles of the presentinvention. Secondly, each Class of MMPGRS Architecture will typicallyhave numerous implementation options, with the illustrative embodimentsshown in FIGS. 8A through 11D1 simply being just a handful of possibleimplementation options.

Thirdly, the MMPGRS Architecture Table set forth in FIG. 7A 1-2 is by nomeans a list of all possible Class of MMPGRS Architecture, but rather isan exemplary listing of the primary classes which comes to mind of theInventors at the time of filing the present Application, and that it isexpected, in the future, that other architectural spaces will evolve orbe developed, thereby providing additional environments in which theMMPGRS of the present invention may be embodied or otherwise practiced.Various examples of how the MMPGRS of the present invention can bepracticed will be described in greater detail below.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation of MultipleGPUs Supported On External Graphics Cards

In FIG. 7A 2, the first illustrative embodiment of the MMPGRS of presentinvention 700 is shown embodied within the HMS Class of MMPGRSArchitecture described in FIG. 7A 1-2. As shown, the Automatic ModeControl Module (AMCM) 400 and the Decomposition, Distribution andRecomposition Modules 401, 402, 403, respectively, of the MultimodeParallel Graphics Rendering Subsystem resides as a software package 701in the Host Memory Space (HMS) while multiple GPUs are supported on apair of external graphic cards 204, 205 connected to a North memorybridge chip (103) and driven in a parallelized manner by the modules ofthe multi-mode parallel graphics rendering subsystem, under the controlof the AMCM. During operation, (i) the Decomposition Module 401 divides(i.e. splits up) the stream of graphic commands and data (GCAD)according to the required parallelization mode, operative at any instantin time, (ii) the Distribution Module 402 uses the North bridge chip todistribute graphic commands and data (GCAD) to the multiple GPUs onboard the external graphics cards, (iii) the Recomposition Module 403uses the North bridge chip to transfer composited pixel data (CPD)between the Recomposition Module (or CPU) and the multiple GPUs duringthe image recomposition stage, and (iv) finally recomposited pixel datasets are displayed as graphical images on one or more display devicesconnected to the external graphics cards via a PCI-express interface,which is connected to the North bridge chip.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation of GPUSupported On Integrated Graphics Device (IGD) and Multiple GPUsSupported on External Graphics Cards

In FIG. 7A 3, the second illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the HMS+IGD Class of MMPGRSArchitecture described in FIG. 7A 1-2. As shown, the Automatic ModeControl Module (AMCM) 400 and the Decomposition, Distribution andRecomposition Modules 401, 402, 403, respectively, of the MultimodeParallel Graphics Rendering Subsystem reside as a software package 701in the Host or CPU Memory Space (HMS), while multiple GPUs are supportedin an IGD within the North memory bridge circuit as well as on externalgraphic cards connected to the North memory bridge chip and driven in aparallelized manner by the modules of the multi-mode parallel graphicsrendering subsystem, under the control of the AMCM. During operation (i)the Decomposition Module 401 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (ii) theDistribution Module 402 uses the North bridge chip to distribute thegraphic commands and data (GCAD) to the multiple GPUs located in the IGDand on the external graphics cards, (iii) the Recomposition Module 403uses the North bridge chip to transfer composited pixel data (CPD)between the Recomposition Module (or CPU) and the multiple GPUs duringthe image recomposition stage, and (iv) finally recomposited pixel datasets are displayed as graphical images on one or more display devicesconnected to one of the external graphics cards or the IGB, as shown.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation of GPUSupported on Integrated Graphics Device (IGD) and Multiple GPUsSupported On External Graphics Cards

In FIG. 7A 4, the third MMPGRS of present invention is shown embodiedwithin the HMS+IGD Class of MMPGRS Architecture described in FIG. 7A1-2. As shown, the Automatic Mode Control Module (AMCM) 400 and theDecomposition, Distribution and Recomposition Modules 401, 402, 403,respectively, of the Multimode Parallel Graphics Rendering Subsystemreside as a software package 701 in the Host Memory Space (HMS) whilemultiple GPUs are supported in an IGD within the South bridge circuit aswell as on external graphic cards connected to the South bridge chip,and driven in a parallelized manner by the modules of the multi-modeparallel graphics rendering subsystem, under the control of the AMCM.During operation, (i) the Decomposition Module 401 divides (i.e. splitsup) the stream of graphic commands and data (GCAD) according to therequired parallelization mode, operative at any instant in time, (ii)the Distribution Module 402 uses the North bridge chip to distributegraphic commands and data (CGAD) to the multiple GPUs located in the IGDand on external graphics cards, (iii) the Recomposition Module 403 usesthe South bridge chip to transfer recomposited pixel data between theRecomposition Module (or CPU) and the multiple GPUs during the imagerecomposition stage, and (iv) finally recomposited pixel data sets aredisplayed as graphical images on one or more display devices connectedto one of the external graphics cards or the IGB, as shown.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation of GPUSupported on Hybrid CPU/GPU Fusion Chip and GPUs Supported on ExternalGraphics Cards

In FIG. 7A 5, the fourth illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the HMS+Fusion Class ofMMPGRS Architecture described in FIG. 7A 1-2. As shown, the AutomaticMode Control Module (AMCM) 400 and the Decomposition, Distribution andRecomposition Modules 401, 402, 403, respectively, of the MultimodeParallel Graphics Rendering Subsystem resides as a software package 701in the Host Memory Space (HMS) while a single GPU (1242) is supported ona CPU/GPU fusion-architecture processor die (alongside the CPU 1241) andone or more GPUs are supported on an external graphic card connected tothe CPU processor die and driven in a parallelized manner by the modulesof the multi-mode parallel graphics rendering subsystem, under thecontrol of the AMCM. During operation, (i) the Decomposition Module 401divides (i.e. splits up) the stream of graphic commands and data (GCAD)according to the required parallelization mode, operative at any instantin time, (ii) the Distribution Module 402 uses the memory controller andinterconnect (e.g. crossbar switch) within the CPU/GPU processor chip todistribute graphic commands and data to the multiple GPUs on the CPU/GPUdie chip and on the external graphics cards, (iii) the RecompositionModule 403 uses the memory controller and interconnect (e.g. crossbarswitch) within the CPU/GPU processor chip to transfer composited pixeldata (CPD) between the Recomposition Module (or CPU) and the multipleGPUs during the image recomposition stage, and (iv) finally recompositedpixel data sets are displayed as graphical images on one or more displaydevices connected to the external graphics card via a PCI-expressinterface, which is connected to the CPU/GPU fusion-architecture chip.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation of MultipleGraphics Pipelines Supported on Multi-Core CPU Chip

In FIG. 7A 6, the fifth illustrative embodiment of the MMPGRS of presentinvention is shown embodied within the HMS+Multicore Class of MMPGRSArchitecture described in FIG. 7A 1-2. As shown, the Automatic ModeControl Module (AMCM) 400 and the Decomposition, Distribution andRecomposition Modules 401, 402, 403, respectively of the MultimodeParallel Graphics Rendering Subsystem resides as a software package 701in the Host or CPU Memory Space (HMS) while some of the CPU cores on amulti-core CPU chip are used to implement a plurality of multi-coregraphics pipelines parallelized by the modules of the software package701 of the multi-mode parallel graphics rendering subsystem, under thecontrol of the AMCM. During operation, (i) the Decomposition Module 401divides (i.e. splits up) the stream of graphic commands and data (GCAD)according to the required parallelization mode, operative at any instantin time, (ii) the Distribution Module 402 uses the North memory bridgeand interconnect network within the multi-core CPU chip to distributegraphic commands and data (GCAD) to the multi-core graphic pipelinesimplemented on the multi-core CPU chip, (iii) the Recomposition Module403: uses the North memory bridge and interconnect network within themulti-core CPU chip to transfer composited pixel data (CPD) between theRecomposition Module (or CPU) and the multi-core graphics pipelinesduring the image recomposition stage, and (iv) finally recompositedpixel data sets are displayed as graphical images on one or more displaydevices connected to the North bridge chip via a display interface.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation of MultipleGPUs Supported on External Graphics Cards, and Carrying Out Pixel ImageRecomposition within External GPUs

In FIG. 7A 7, the sixth illustrative embodiment of the MMPGRS of presentinvention is shown embodied within the HMS+GPU-Recomposition Class ofMMPGRS Architecture described in FIG. 7A 1-2, and in copending U.S.patent application Ser. No. 11/648,160 incorporated herein by reference.As shown, the Automatic Mode Control Module (AMCM) 400 and theDecomposition, Distribution and Recomposition Modules 401, 402respectively, of the Multimode Parallel Graphics Rendering Subsystemresides as a software package 701 in the Host or CPU Memory Space (HMS)while multiple GPUs on external GPU cards driven in parallelized mannerby the modules of the software package 701 of the multi-mode parallelgraphics rendering subsystem, under the control of the AMCM, and two ormore GPUs 715, 716 are used to implement the Recomposition Module.During operation, (i) the Decomposition Module 401 divides (i.e. splitsup) the stream of graphic commands and data (GCAD) according to therequired parallelization mode, operative at any instant in time, (ii)the Distribution Module 402 uses the North or South bridge circuit andinterconnect network to distribute graphic commands and data (GCAD) tothe external GPUs, (ii) the Recomposition Module uses the North memorybridge and associated system bus (e.g. PCI-express bus) to transfercomposited pixel data (CPD) between the GPUs during the imagerecomposition stage, and (iv) finally recomposited pixel data sets aredisplayed as graphical images on one or more display devices, connectedto an external graphics card via a PCI-express interface which isconnected to either the North or South bridge circuits of the hostcomputing system.

During the Time Division Mode of this MMPGRS, each non-primary GPU,during its assigned time slot, moves its full-color composited image tothe frame buffer (FB) of the primary GPU for display on the displayscreen of the display device.

Considering the case of a dual-GPU MMPGRS, for simplicity ofexplanation, during the Image Division Mode of this MMPGRS, the primaryGPU outputs one/half of the color image in its frame buffer (FB) to thedisplay device, while the secondary GPU moves one/half of the image inits FB to the primary GPU. Then the primary GPU does the same with thesecond part of the image in its frame buffer (FB). Thus, during theImage Division Mode, the recomposition involves a coordinated output oftwo image halves, that have been composited within the frame buffers(FBs) of two GPUs, to the frame buffer of the primary GPU (forrecompositing) and ultimately display to the display device. In thismode, there is no merge function between the pixels of these two imagehalves involved in the recompositing process. In the case of multipleGPUs, the process is essentially the same, except that each GPU movesits 1/n part of the image to the frame buffer of the primary GPU forrecompositing and subsequent display.

During the Object Division Mode of this MMPGRS, the merging of pixeldata in the FBs of the GPUs is much more complicated, involvingcompositing within the vertex and/or fragment shaders of the primaryGPU, as will be described in great detail below.

Referring to FIG. 7A 7-1, the innovative pixel recompositing processsupported within the MMPGRS of FIG. 7A 7 during its Object Division Modewill now be described in great technical detail.

In general, the recompositing phase/stage of the present inventioninvolves moving the pixel Depth and Color values from the frame buffers(FB) in the secondary GPPL, to the FB in the primary GPPL (viainter-GPPL communication), and then merging these pixel values withtheir counterparts at the primary GPPL by means of programmable FragmentShader supported in the pixel processing subsystem (211). FIG. 7A 7-2describes the compositing process carried out by programmable FragmentShader for the case where the MMPGRS employs dual GPPLs (i.e. GPUs). Itis understood, however, that if more GPPLs are involved, then the(re)compositing process will repeat accordingly for each additional“secondary” GPPL, until the final step when the partially compositedpixel data in the frame buffer (FB) in the last secondary GPPL, isfinally recomposited within the pixel data within the frame buffer (FB)of the primary GPPL.

As shown in FIG. 7A 7-1, the pixel frame generating pipeline includesthree basic steps, namely: the decompose 402, distribute 403, and render404 stages. Towards the end of the graphics processing pipeline (GPPL),the recompose step 405 is carried out for final FB, which is finallydisplayed to the display device 405.

During the Decompositing step 402, graphics commands and data stream aredecomposed into well load balanced sub-streams in the DecompositingModule 504, keeping state consistency of the graphics libraries.

The Distributing step 403 is supervised by the Distribution module 505.Decomposed graphics commands and data elements are sent to the Vendor'sGPU Driver (506) and memory bridge (203), and delivered for renderingthe primary 205 and secondary 204 graphics cards, via separatePCIexpress buses 207, 208.

Rendering (step 404) is done simultaneously 602, 603 in both GPPLs,creating two partial FBs. The compositing process (step 405) comprisesthe following substeps:

-   -   Step (606): The color FB is read back from the secondary GPPL,        and moved via memory bridge (203) to the primary GPPL's Texture        memory (218) as a texture tex1.    -   Step (607)L The Z-buff is read back from the secondary GPPL, and        moved via memory bridge (203) to the primary GPPL's Texture        memory (218) as a texture dep1.    -   Step (604): Color FB of primary GPPL is copied to texture memory        as texture tex2.    -   Step (605): Z-buffer of primary GPPL is copied to texture memory        as texture dep2.    -   Step (608): Shader code for recomposition (described in FIG. 7B        7-2) is downloaded and exercised on four textures tex1, tex2,        dep1, dep2 as follows:    -   Step (609): The two depth textures are compared pixel by pixel        for their depth values.    -   Assuming the rule that the closest pixel is the one to be        transferred to the final FB, at each x,y location the two depth        textures are compared for lowest depth value, the lowest is        chosen, and the color value at x,y of its correspondent color        texture is moved to the x,y location in the final texture.    -   Step (610): The resulting texture is copied back to the primary        color FB.    -   To complete rendering (step 404 b), the following substeps are        performed:    -   Step (611): All transparent objects of the scene and overlays        (such as score titles) are essentially kept by applications for        the very last data to be rendered. Therefore, once all opaque        objects have been rendered in parallel at separate GPPLs and        composed back to the primary's FB, the additional and final        phase of a non-parallel rendering of transparent objects takes        place in the primary GPPL.    -   Step (612): The final FB is sent to the display device for        display on its display screen.

In step 405, the detailed shader program is used to composite two colortextures based on the depth test conducted between the two depthtextures, as shown in FIG. 7B 7-2.

While the above illustrative embodiment discloses the use of theFragment Shader in the pixel processing subsystem/engine within theprimary GPPL, to carry out the composition process in the dualGPPL-based graphics platform of the present invention, it is understoodthat other computational resources within the GPPL can be used inaccordance with the scope and spirit of the present invention. Inparticular, in an alternative illustrative embodiment, the recompositingphase/stage can involve moving the pixel Depth and Color values from theframe buffers (FBs) in the secondary GPPLs, to the FB in the primaryGPPL (via inter-GPPL communication), and then merging these pixel valueswith their counterparts at the primary GPPL by means of the programmableVertex Shader provided in the geometry processing subsystem 210 of theprimary GPPL. And in yet another illustrative embodiment of the presentinvention, the recompositing phase/stage can involve moving the pixelDepth and Color values from the frame buffers FB in the secondary GPPLs,to the FB in the primary GPPL (via inter-GPPL communication), and thenmerging these pixel values with their counterparts at the primary GPPLby the means of both programmable Vertex and Fragment Shaders providedin the geometry and pixel processing subsystems in the primary GPPL.Such modifications will become readily apparent to those skilled in theart having the benefit of the present inventive disclosure.

In the general case of MMPGRS having n GPPLs, the pixel data containedin the Frame Buffers (FB) associated with the secondary GPPLs are movedto the primary GPPL by way of an inter-GPPL communication process (e.g.the Interconnect network 404 implemented by multiple-lane PCI Express™buses), and then processed within the local FB of the primary GPPL, toperform pixel image (re)composition. The pixel composition result isthen sent to display device, and alternatively, also returned to thesecondary GPPLs, if required in some applications as a basis of the nextpixel frame.

As shown in FIGS. 7A7, 7A7-1 and 7A7-2 and described in detail above,the GPPL-based recomposition process of the present invention can beimplemented across two or more GPPLs using software that (i) controlsthe computational machinery within the GPPLs of the MMPGRS platform, and(ii) exploits the Shader (pixel) processing capabilities in the primaryGPPL, with no need for any external hardware. Alternatively, however,the GPPL-based recomposition process of the present invention can beimplemented across two or more GPPLs using hardware circuitry and/orfirmware (within a graphics hub architecture of the present invention)that (i) controls the computational machinery within the GPPLs of theMMPGRS platform, and (ii) exploits the Shader (pixel) processingcapabilities in the primary GPPL, as shown in FIGS. 7B2, 7B4-1, 7B6-1,7B7-1, 7B8-3, and 7B1, and described below.

FIG. 7A 7-3 illustrates the time-line of one complete composited pixelframe, including time slots associated with the different steps ofobject division rendering. As shown, the reuse of GPPL resources forrecompositing occurs during a time slot, where the GPPL resources aregenerally idle during the recompose step. Thus, by virtue of the presentinvention, GPPL resources are used “for free” during recomposition,without sacrificing system performance.

The Graphics Hub Structure of the Present Invention Expressed inDifferent Ways in Different MMPGRS System Architectures

While FIGS. 7B1 through 7B11 illustrate that the graphics hub device(GHD) of the present invention can be expressed in different ways indifferent MPGRS system architectures, it should be pointed out that,within each such system architecture, the function of the graphics hubdevice (GHD) is essentially the same, namely: (i) to interconnect thegraphics-based application in memory space, with the cluster of GPUs orCPU-cores along the parallelized GPPLs; and (ii) to support the basicfunctionalities of the Distribution Module 402 and the RecompositionModule 403 in such MMPGRS system architectures.

Also, it should be noted that from a functional point of view, theDistribution Module resides before the cluster of GPUs or CPU-cores,delivering graphics commands and data (GCAD) for rendering (and thusfunctioning as a “pre GPU unit” of sorts), whereas the RecompositionModule functions logically after the cluster of GPUs, and collects postrendering data (“post GPU unit”). However, both the Distribution Moduleand the Recomposition Module typically share the same physical hardwareunit (e.g. silicon chip). Various examples of the graphics hub device(GHD) of the present invention will now be described in great detail inFIGS. 7B1 through 7B11, for various types of MMGPRS system architecturesindicated in FIG. 7A 1-2.

At this juncture, the two major advantages should be pointed out whenusing the “graphics hub device” architecture approach of the presentinvention, illustrated in FIGS. 7B1 through 7B11.

The first advantage of the “graphics hub device” architecture is thatthe number of driven GPPLs in the MMPGRS is no longer limited by thenumber of buses provided by the memory bridge circuit employed in theCPU-based host computing system. The Interconnect Network 404 employedin the graphics hub device (GD) of the present invention allows(theoretically) for the connection of an unlimited number of GPUs to theHost CPU.

The second advantage of the “graphics hub device” architecture is thehigh performance achieved during image recomposition using the graphichub device architecture, thereby eliminating the need of moving theFrame Buffer (FB) pixel data from multiple GPPLs to the host or CPUmemory for merging, as it is done in the system architecturesillustrated in FIGS. 7A2 through 7A7. During GPU-based Recompositionprocess of the present invention, the merge function is performed byfast, highly specialized hardware within the GPUS, independent of othertasks that are concurrently trying to access the main memory of the hostcomputing system, which occurs in a multi-tasking system architecturesillustrated in FIGS. 7A2 through 7A7.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation of MultipleGPUs Supported on External Graphics Cards Connected to Graphics HubDevice of the Present Invention

In FIG. 7B 1, the seventh illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the Hub Class of MMPGRSArchitecture described in FIG. 7A 1-2. As shown, the Automatic ModeControl Module (AMCM) 400 and the Decomposition Submodule No. 1 401′reside as a software package in the Host or CPU Memory Space (HMS) whilethe Decomposition Submodule No. 2 401″, Distribution Module 402″ andRecomposition Module 403″ are realized within a single graphics hubdevice (e.g. chip) that is connected to the North memory bridge of thehost computing system via a PCI-express interface and to a cluster ofexternal GPUs 410″ via an interconnect, with the GPUs being driven in aparallelized manner by the modules of the multi-mode parallel graphicsrendering subsystem, under the control of the AMCM, (ii) theDecomposition Submodule No. 1 transfers graphic commands and data (GCAD)to the Decomposition Submodule No. 2 via the North memory bridgecircuit, (iii) the Decomposition Submodule No. 2 divides (i.e. splitsup) the stream of graphic commands and data (GCAD) according to therequired parallelization mode, operative at any instant in time, (iv)the Distribution Module 402″ distributes graphic commands and data(GCAD) to the external GPUs, (v) the Recomposition Module 403′ transferscomposited pixel data (CPD) between the GPUs during the imagerecomposition stage, and (vi) finally recomposited pixel data sets aredisplayed as graphical images on one or more display devices connectedto the primary GPU on the graphical display card which is connected tothe graphics hub chip of the present invention via the interconnect404″.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation Multiple GPUsSupported on External Graphics Cards and Carrying Out ImageRecomposition Across Two or More of Said GPUs

In FIG. 7B 2, the eighth illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the Hub+GPU-RecompositionClass of MMPGRS Architecture described in FIG. 7A 1-2. As shown, theAutomatic Mode Control Module (AMCM) 400 and the Decomposition SubmoduleNo. 1 401′ reside as a software package in the Host Memory Space (HMS)of the host computing system, while the Decomposition Submodule No. 2401″ and the Distribution Module 402″ are realized within a singlegraphics hub device (e.g. chip) that is connected to the North bridge ofthe host computing system and a cluster of external GPUs 410″, and theRecomposition Module 403″ is implemented across two or more GPUs 715,716, as taught in FIG. 7A 7, and that all of the GPUs are driven in aparallelized manner, under the control of the AMCM. During operation,(i) the Decomposition Submodule No. 1 transfers graphic commands anddata (GCAD) to the Decomposition Submodule No. 2 via the North bridgecircuit, (ii) the Decomposition Submodule No. 2 divides (i.e. splits up)the stream of graphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iii) theDistribution Module 402″ distributes graphic commands and data (GCAD) tothe external GPUs, (iv) the Recomposition Module 403″, implementedwithin the primary GPU 715, transfers composited pixel data (CPD)between the GPUs during the image recomposition stage, and (v) finallyrecomposited pixel data sets (recomposited within the vertex and/orfragment shaders of the primary GPU) are displayed as graphical imageson one or more display devices connected to the primary GPU on thegraphical display card(s), which are connected to the graphics hub chipof the present invention.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation Multiple GPUsSupported on an Integrated Graphics Device (IGD) within a North MemoryBridge Chip

In FIG. 7B 3, the ninth illustrative embodiment of the MMPGRS of presentinvention is shown embodied within the Chipset Class of MMPGRSArchitecture described in FIG. 7A 1-2. As shown, the Automatic ModeControl Module (AMCM) 400 and the Decomposition Submodule No. 1 401′reside as a software package in the Host or CPU Memory Space (HMS) whilethe Decomposition Submodule No. 2 401″, Distribution Module 402″ andRecomposition Module 403″ are realized (as a graphics hub) in anintegrated graphics device (IGD) within the North memory bridge circuitand having a plurality of GPUs being driven in a parallelized manner bythe modules of the multi-mode parallel graphics rendering subsystem,under the control of the AMCM. During operation, (i) the DecompositionSubmodule No. 1 transfers graphic commands and data (GCAD) to theDecomposition Submodule No. 2 via the North bridge circuit, (ii) theDecomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iii) theDistribution Module 402″ distributes graphic commands and data (GCAD) tothe internal GPUs via the interconnect network, (iv) the RecompositionModule 403″ transfers composited pixel data (CPD) between the GPUsduring the image recomposition stage, and (v) finally recomposited pixeldata sets are displayed as graphical images on one or more displaydevices connected to the external graphical display card, or the primaryGPU in the IGB, as shown.

Illustrative Embodiment of the MMPGRS Of The Present Invention Having aSystem Architecture Capable of Parallelizing the Operation Multiple GPUsSupported on an Integrated Graphics Device (IGD) within a SouthBridge-Chip

In FIG. 7B 4, the tenth illustrative embodiment of the MMPGRS of presentinvention is shown embodied within the Chipset Class of MMPGRSArchitecture described in FIG. 7A 1-2. As shown, the Automatic ModeControl Module (AMCM) 400 and the Decomposition Submodule No. 1 401′reside as a software package in the Host or CPU Memory Space (HMS) whilethe Decomposition Submodule No. 2 401″, Distribution Module 402″ andRecomposition Module 403″ are realized (as a graphics hub) in anintegrated graphics device (IGD) within the South bridge circuit of thehost computing system and having a plurality of GPUs driven in aparallelized manner by the modules of the multi-mode parallel graphicsrendering subsystem, under the control of the AMCM. During operation,(i) the Decomposition Submodule No. 1 transfers graphic commands anddata (GCAD) to the Decomposition Submodule No. 2 via the communicationinterfaces of the North and South bridge circuits, (ii) theDecomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iii) theDistribution Module 402″ distributes graphic commands and data (GCAD) tothe external GPUs, (iv) the Recomposition Module 403″ transferscomposited pixel data (CPD) between the GPUs during the imagerecomposition stage, and (v) finally recomposited pixel data sets aredisplayed as graphical images on one or more display devices connectedto the external graphical display card, or the primary GPU in the IGB,as shown.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation Multiple GPUsSupported on an Integrated Graphics Device (IGD) within a South BridgeChip, Wherein Recomposition is Implemented Across Two or More GPUs

In FIG. 7B 4-1, the eleventh illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the Chipset Class of MMPGRSArchitecture described in FIG. 7A 1-2. As shown, the Automatic ModeControl Module (AMCM) 400 and the Decomposition Submodule No. 1 401′reside as a software package in the Host or CPU Memory Space (HMS) whilethe Decomposition Submodule No. 2 401″ and the Distribution Module 402″403″ are realized (as a graphics hub) in an integrated graphics device(IGD) within the South bridge circuit of the host computing system andhaving a plurality of GPUs driven in a parallelized manner by themodules of the multi-mode parallel graphics rendering subsystem, underthe control of the AMCM, while the Recomposition Module 403″ isimplemented across two or more GPUs 715, 716. During operation, (i) theDecomposition Submodule No. 1 transfers graphic commands and data (GCAD)to the Decomposition Submodule No. 2 via the communication interfaces ofthe North and South bridge circuits, (ii) the Decomposition SubmoduleNo. 2 divides (i.e. splits up) the stream of graphic commands and data(GCAD) according to the required parallelization mode, operative at anyinstant in time, (iii) the Distribution Module 402″ distributes graphiccommands and data (GCAD) to the external GPUs, (iv) the RecompositionModule 403″, implemented at the Primary GPU, transfers composited pixeldata (CPD) between the GPUs during the image recomposition stage, and(v) finally recomposited pixel data sets are displayed as graphicalimages on one or more display devices connected to the externalgraphical display card, or the primary GPU in the IGB, as shown.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation of MultipleGPUs Supported on an Integrated Graphics Device (IGD) within a NorthMemory Bridge Chip, and GPUs on an External Graphics Card

In FIG. 7B 5, the twelfth illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the Chipset Class of MMPGRSArchitecture described in FIG. 7A 1-2. As shown, the Automatic ModeControl Module (AMCM) 400 and the Decomposition Submodule No. 1 401′reside as a software package in the Host or CPU Memory Space (HMS) whilethe Decomposition Submodule No. 2 401″, Distribution Module 402″ andRecomposition Module 403″ are realized (as a graphics hub) in anintegrated graphics device (IGD) within the North memory bridge of thehost computing system and having multiple GPUs being driven with asingle GPU on an external graphics card in a parallelized manner by themodules of the multi-mode parallel graphics rendering subsystem, underthe control of the AMCM. During operation, (i) the DecompositionSubmodule No. 1 transfers graphics commands and data (GCAD) to theDecomposition Submodule No. 2 via the North bridge circuit, (ii) theDecomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iii) theDistribution Module 402″ distributes graphic commands and data (GCAD) tothe external GPUs, (iv) the Recomposition Module 403″ transferscomposited pixel data (CPD) between the GPUs during the imagerecomposition stage, and (v) finally recomposited pixel data sets aredisplayed as graphical images on one or more display devices connectedto the external graphical display card, or the primary GPU in the IGB,as shown.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation of a SingleGPU Supported on an Integrated Graphics Device (IGD) within a SouthBridge Chip, and Multiple GPUs Supported on an External Graphics Card

In FIG. 7B 6, the thirteenth illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the Chipset Class of MMPGRSArchitecture described in FIG. 7A 1-2. As shown, the Automatic ModeControl Module (AMCM) 400 and the Decomposition Submodule No. 1 401′reside as a software package in the Host or CPU Memory Space (HMS) whilethe Decomposition Submodule No. 2 401″, Distribution Module 402″ andRecomposition Module 403″ are realized (as a graphics hub) in anintegrated graphics device (IGD) within the South bridge circuit of thehost computing system and having multiple GPUs driven with a single GPUon an external graphics card in a parallelized manner by the modules ofthe multi-mode parallel graphics rendering subsystem, under the controlof the AMCM. During operation, (i) the Decomposition Submodule No. 1transfer graphic commands and data (GCAD) to the Decomposition SubmoduleNo. 2 via the North and South bridge circuits, (ii) the DecompositionSubmodule No. 2 divides (i.e. splits up) the stream of graphic commandsand data (GCAD) according to the required parallelization mode,operative at any instant in time, (iii) the Distribution Module 402″distributes the graphic commands and data (GCAD) to the external GPUs,(iv) the Recomposition Module 403″ transfers composited pixel data (CPD)between the GPUs during the image recomposition stage, and (v) finallyrecomposited pixel data sets are displayed as graphical images on one ormore display devices connected to the external graphics card, or theprimary GPU in the IGB, as shown.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation of a SingleGPU Supported on an Integrated Graphics Device (IGD) within a SouthBridge Chip, and Multiple GPUs Supported on an External Graphics Cardwith the Recomposition Module Implemented Across Two or More GPUs

In FIG. 7B 6-1, the fourteenth illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the Chipset Class of MMPGRSArchitecture described in FIG. 7A 1-2. As shown, the Automatic ModeControl Module (AMCM) 400 and the Decomposition Submodule No. 1 401′reside as a software package in the Host or CPU Memory Space (HMS) whilethe Decomposition Submodule No. 2 401″ and Distribution Module 402″ arerealized (as a graphics hub) in an integrated graphics device (IGD)within the South bridge circuit of the host computing system and havingmultiple GPUs driven with a single GPU on an external graphics card in aparallelized manner by the modules of the multi-mode parallel graphicsrendering subsystem, under the control of the AMCM, while theRecomposition Module 403″ is implemented across two or more GPUs 715,716. During operation, (i) the Decomposition Submodule No. 1 transfergraphic commands and data (GCAD) to the Decomposition Submodule No. 2via the North and South bridge circuits, (ii) the DecompositionSubmodule No. 2 divides (i.e. splits up) the stream of graphic commandsand data (GCAD) according to the required parallelization mode,operative at any instant in time, (iii) the Distribution Module 402″distributes the graphic commands and data (GCAD) to the external GPUs,(iv) the Recomposition Module 403″ transfers composited pixel data (CPD)between the GPUs during the image recomposition stage, and (v) finallyrecomposited pixel data sets are displayed as graphical images on one ormore display devices connected to the external graphics card, or theprimary GPU in the IGB, as shown.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Employing an Integrated Graphics Device (IGD) withina Bridge Chip Capable of Parallelizing the Operation of Multiple GPUsSupported on Multiple External Graphics Cards, or Controlling a SingleGPU within the IGD of the Present Invention for Driving an DisplayDevice Connected Thereto

In FIG. 7B 7, the fifteenth illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the following the ChipsetClass of MMPGRS Architecture described in FIG. 7A 1-2. The shown, theAutomatic Mode Control Module (AMCM) 400 and the Decomposition SubmoduleNo. 1 401′ reside as a software package in the Host or CPU Memory Space(HMS) while the Decomposition Submodule No. 2 401″, Distribution Module402″ and Recomposition Module 403″ are realized (as a graphics hub) inan integrated graphics device (IGD) realized within the North memorybridge chip of the host computing system, and driving (i) multiple GPUson multiple external graphics cards in a parallelized manner by themodules of the multi-mode parallel graphics rendering subsystem, underthe control of the AMCM, or alternatively (ii) controlling a single GPUaboard the IGD for driving a display device connected to the IGD via adisplay interface. During operation, (i) the Decomposition Submodule No.1 transfers graphic commands and data (GCAD) to the DecompositionSubmodule No. 2 via the North bridge circuit, (ii) the DecompositionSubmodule No. 2 divides (i.e. splits up) the stream of graphic commandsand data (GCAD) according to the required parallelization mode,operative at any instant in time, (iii) the Distribution Module 402″distributes the graphic commands and data (GCAD) to the internal GPU andexternal GPUs, (iv) the Recomposition Module 403″ transfers compositedpixel data (CPD) between the GPUs during the image recomposition stage,and (v) finally recomposited pixel data sets are displayed as graphicalimages on one or more display devices connected to one of the externalgraphic cards or the primary GPU in the IGB, as shown.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Employing an Integrated Graphics-Device (IGD) withina Bridge Chip Capable of (i) Parallelizing the Operation of MultipleGPUs Supported on Multiple External Graphics Cards with theRecomposition Module Implemented Across Two or More GPUs, or (ii)Controlling a Single GPU Within the IGD of the Present Invention forDriving an Display Device Connected Thereto

In FIG. 7B 7-1, the sixteenth illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the following the ChipsetClass of MMPGRS Architecture described in FIG. 7A 1-2. The shown, theAutomatic Mode Control Module (AMCM) 400 and the Decomposition SubmoduleNo. 1 401′ reside as a software package in the Host or CPU Memory Space(HMS) while the Decomposition Submodule No. 2 401″ and DistributionModule 402″ are realized (as a graphics hub) in an integrated graphicsdevice (IGD) realized within the North memory bridge chip of the hostcomputing system, and driving (i) multiple GPUs on multiple externalgraphics cards in a parallelized manner by the modules of the multi-modeparallel graphics rendering subsystem, under the control of the AMCM, oralternatively (ii) controlling a single GPU aboard the IGD for driving adisplay device connected to the IGD via a display interface, while theRecomposition Module 403″ is implemented across two or more GPUs (715,716). During operation, (i) the Decomposition Submodule No. 1 transfersgraphic commands and data (GCAD) to the Decomposition Submodule No. 2via the North bridge circuit, (ii) the Decomposition Submodule No. 2divides (i.e. splits up) the stream of graphic commands and data (GCAD)according to the required parallelization mode, operative at any instantin time, (iii) the Distribution Module 402″ distributes the graphiccommands and data (GCAD) to the internal GPU and external GPUs, (iv) theRecomposition Module 403″ transfers composited pixel data (CPD) betweenthe GPUs during the image recomposition stage, and (v) finallyrecomposited pixel data sets are displayed as graphical images on one ormore display devices connected to one of the external graphic cards orthe primary GPU in the IGB, as shown.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Employing a CPU/GPU Fusion-Type Chip Capable ofParallelizing the Operation of an Internal GPU and Multiple GPUsSupported on an External Graphics Card

In FIG. 7B 8-1, the seventeenth illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the following the CPU/GPUFusion Class of MMPGRS Architecture described in FIG. 7A 1-2. As shown,the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host MemorySpace (HMS) while the Decomposition Submodule No. 2 401″, DistributionModule 402″ and Recomposition Module 403″ are realized (as a graphicshub) in on the die of a hybrid CPU/GPU fusion-architecture chip withinthe host computing system and having a single GPU driven with one ormore GPUs on an external graphics card (connected to the CPU/GPU chip)in a parallelized manner by the modules of the multi-mode parallelgraphics rendering subsystem under the control of the AMCM. Duringoperation, (i) the Decomposition Submodule No. 1 transfers graphicscommands and data (GCAD) to the Decomposition Submodule No. 2, (ii) theDecomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iii) theDistribution Module 402″ distributes the graphic commands and data(GCAD) to the internal GPU and external GPUs, (iv) the RecompositionModule 403″ transfers composited pixel data (CPD) between the GPUsduring the image recomposition stage, and (v) finally recomposited pixeldata sets are displayed as graphical images on one or more displaydevices 106 connected to the external graphics card connected to thehybrid CPU/GPU chip via a PCI-express interface.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Employing A CPU/GPU Fusion-Type Chip Capable ofParallelizing the Operation of Multiple Internal GPUs And Multiple GPUsSupported on an External Graphics Card

In FIG. 7B 8-2, the eighteenth illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the following the CPU/GPUFusion Class of MMPGRS Architecture described in FIG. 7A 1-2. The shown,the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host MemorySpace (HMS) while the Decomposition Submodule No. 2 401″, DistributionModule 402″ and Recomposition Module 403″ are realized (as a graphicshub) on the die of a multi-core CPU chip within the host computingsystem and having multiple CPU cores, some of which implement multiplesoft parallel graphics pipelines (“soft GPUs”) driven in a parallelizedmanner by the modules of the multi-mode parallel graphics renderingsubsystem under the control of the AMCM. During operation, (i) theDecomposition Submodule No. 1 transfers graphics commands and data(GCAD) to the Decomposition Submodule No. 2 via the North memory bridgecircuit and interconnect network within the multi-core CPU chip, (ii)the Decomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iii) theDistribution Module 402″ uses the crossbar switch (i.e. interconnect) onthe processor die to distribute the graphic commands and data (GCAD) tothe multiple soft parallel graphics pipelines (implemented by themultiple CPU cores), (v) the Recomposition Module 403″ transferscomposited pixel data (CPD) between the multiple CPU cores during theimage recomposition stage, and (vi) finally recomposited pixel data setsare displayed as graphical images on one or more display devices 106connected to the North memory bridge chip via a display interface.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Employing a CPU/GPU Fusion-Type Chip Capable ofParallelizing the Operation of Multiple Internal GPUs and Multiple GPUsSupported on an External Graphics Card, with the Recomposition ModuleBeing Implemented Across Two of More of Said GPUs

In FIG. 7B 8-3 the nineteenth illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the following the CPU/GPUFusion Class of MMPGRS Architecture described in FIG. 7A 1-2. The shown,(i) the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ reside as a software package in the Host MemorySpace (HMS), (ii) the Decomposition Submodule No. 2 401″ andDistribution Module 402″ are realized (as a graphics hub) in on the dieof a hybrid CPU/GPU fusion-architecture chip within the host computingsystem and having multiple GPUs 1242″ driven with one or more GPUs on anexternal graphics card 205 (connected to the CPU/GPU chip) in aparallelized manner by the modules of the multi-mode parallel graphicsrendering subsystem under the control of the AMCM, and (iii) theRecomposition Module 403″ is implemented across two or more GPUs 715,716 provided on the CPU/GPU fusion chip die and external graphics cards.During operation, (iv) the Decomposition Submodule No. 1 transfersgraphics commands and data (GCAD) to the Decomposition Submodule No. 2,(v) the Decomposition Submodule No. 2 divides (i.e. splits up) thestream of graphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (vi) theDistribution Module 402″ uses the crossbar switch (i.e. interconnect) onthe processor die to distribute the graphic commands and data (GCAD) tothe internal GPUs and external GPUs, (vii) the Recomposition Module 403″transfers composited pixel data (CPD) between the GPUs during the imagerecomposition stage, and (viii) finally recomposited pixel data sets aredisplayed as graphical images on one or more, display devices 106connected to the external graphics card connected to the hybrid CPU/GPUchip via a PCI-express interface.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation of MultipleGraphics Pipelines Implemented on a Multi-Core CPU Chip of the PresentInvention and Driving a Display Device Connected to the North MemoryBridge Chip of the Host Computing System

In FIG. 7B 9-1, the twentieth illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the following the MulticoreCPU Class of MMPGRS Architecture described in FIG. 7A 1-2. As shown, theAutomatic Mode Control Module (AMCM) 400 and the Decomposition SubmoduleNo. 1 401′ reside as a software package in the Host Memory Space (HMS)while the Decomposition Submodule No. 2 401″, Distribution Module 402″and Recomposition Module 403″ are realized (as a graphics hub) on thedie of a multi-core CPU chip within the host computing system. As shown,some of the CPU cores are used to implement multiple soft parallelgraphics pipelines (“soft GPUs”) that are driven in a parallelizedmanner by the modules of the multi-mode parallel graphics renderingsubsystem under the control of the AMCM. During operation, (i) theDecomposition Submodule No. 1 transfers graphics commands and data(GCAD) to the Decomposition Submodule No. 2 via the North memory bridgecircuit and interconnect network within the multi-core CPU chip, (ii)the Decomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iii) theDistribution Module 402″ uses the crossbar switch (i.e. interconnect) onthe processor die to distribute the graphic commands and data (GCAD) tothe multiple soft parallel graphics pipelines (implemented by themultiple CPU cores), (iv) the Recomposition Module 403″ transferscomposited pixel data (CPD) between the multiple CPU cores during theimage recomposition stage, and (v) finally recomposited pixel data setsare displayed as graphical images on one or more display devices 106connected to the North memory bridge chip via a display interfaceimplemented therein, as shown.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Capable of Parallelizing the Operation of MultipleSoft Graphics Pipelines Implemented on a Multi-Core CPU Chip, and One orMore GPUs Supported on an External Graphics Card Interfaced to theMulti-Core CPU Chip

In FIG. 7B 9-2, the twenty-first illustrative embodiment of the MMPGRSof present invention is shown embodied within the following theMulticore CPU Class of MMPGRS Architecture described in FIG. 7A 1-2. Theshown, the Automatic Mode Control Module (AMCM) 400 and theDecomposition Submodule No. 1 401′ resides as a software package 711 inthe Host Memory Space (HMS) while the Decomposition Submodule No. 2401″, the Distribution Module 402″ and the Recomposition Module 403″ arerealized as a graphics hub within a multi-core CPU chip employed withinthe host computing system, having a plurality of CPU cores, some ofwhich implement multiple soft graphics pipelines which, along withmultiple GPUs supported on an external graphics card 205, are driven ina parallelized manner by the modules of the multi-mode parallel graphicsrendering subsystem under the control of the AMCM. During operation, (i)the Decomposition Submodule No. 1 transfers graphics commands and data(GCAD) to the Decomposition Submodule No. 2 via the interconnects withinthe North memory bridge chip and the multi-core CPU chip, (ii) theDecomposition Submodule No. 2 divides (i.e. splits up) the stream ofgraphic commands and data (GCAD) according to the requiredparallelization mode, operative at any instant in time, (iii) theDistribution Module 402″ uses the interconnect (i.e. crossbar switch) inthe multi-core CPU chip to distribute the graphic commands and data(GCAD) to the multiple soft graphics pipelines (e.g. soft GPUs) and theGPUs on the external graphics card 205, and (iv) the RecompositionModule 403″ transfers composited pixel data (CPD) between the softgraphics pipelines on the multi-core CPU chip and hard GPUs on theexternal graphics card during the image recomposition stage, and (v)finally recomposited pixel data sets are displayed as graphical imageson one or more display devices 106 connected to the external graphicscard which is connected to the multi-core CPU chip via a PCI-expressinterface.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Employing a Graphics Hub Device Capable ofParallelizing the Operation of Multiple GPUs Supported on a Game ConsoleBoard

In FIG. 7B 10, the twenty-second illustrative embodiment of the MMPGRSof present invention is shown embodied within the following the GameConsole Class of MMPGRS Architecture described in FIG. 7A 1-2. As shown,the Automatic Mode Control Module (AMCM) 400 and the DecompositionSubmodule No. 1 401′ are realized as a software package 711 within theHost Memory Space (HMS), while the Decomposition Submodule No. 2 401″,the Distribution Module 402″ and the Recomposition Module 403′ arerealized as a graphics hub semiconductor chip within the game consolesystem in which multiple GPUs are driven in a parallelized manner by themodules of the multi-mode parallel graphics rendering subsystem underthe control of the AMCM. During operation, (i) the DecompositionSubmodule No. 1 transfers graphics commands and data (GCAD) to theDecomposition Submodule No. 2, via the memory controller on themulti-core CPU chip and the interconnect in the graphics hub chip of thepresent invention, (ii) the Decomposition Submodule No. 2 divides (i.e.splits up) the stream of graphic commands and data (GCAD) according tothe required parallelization mode, operative at any instant in time,(iii) the Distribution Module 402″ distributes the graphic commands anddata (GCAD) to the multiple GPUs, (iv) the Recomposition Module 403″transfers composited pixel data (CPD) between the multiple GPUs duringthe image recomposition stage, and (v) finally recomposited pixel datasets (recomposited within the vertex and/or fragment shaders of theprimary GPU) are displayed as graphical images on one or more displaydevices 106 connected to the primary GPU 715 via a analog displayinterface.

Illustrative Embodiment of the MMPGRS of the Present Invention Having aSystem Architecture Employing a Graphics Hub Device Capable ofParallelizing the Operation of Multiple GPUs Supported on a Game ConsoleBoard, with the Recomposition Module Realized Across Two or More GPUs

In FIG. 7B 11, the twenty-third illustrative embodiment of the MMPGRS ofpresent invention is shown embodied within the following the GameConsole Class of MMPGRS Architecture described in FIG. 7A 1-2. As shown,the Profiling and Control Automatic Mode Control Module (AMCM) 400 andthe Decomposition Submodule No. 1 401′ are realized as a softwarepackage 711 within the Host Memory Space (HMS) of the host computingsystem while the Decomposition Submodule No. 2 401″ and DistributionModule 402′ are realized as a graphics hub semiconductor chip within thegame console system in which multiple GPUs are driven in a parallelizedmanner by the modules of the multi-mode parallel graphics renderingsubsystem under the control of the AMCM, while the Recomposition Module403′ is implemented across two or more GPUs 715, 716. During operation,(i) the Decomposition Submodule No. 1 transfers graphics commands anddata (GCAD) to the Decomposition Submodule No. 2, via the memorycontroller on the multi-core CPU chip and the interconnect in thegraphics hub chip of the present invention, (ii) the DecompositionSubmodule No. 2 divides (i.e. splits up) the stream of graphic commandsand data (GCAD) according to the required parallelization mode,operative at any instant in time, (iii) the Distribution Module 402′distributes the graphic commands and data (GCAD) to the multiple GPUs,(iv) the Recomposition Module 403′, realized primarily within thesubstructure of the primary GPU, transfers composited pixel data (CPD)between the multiple GPUs during the image recomposition stage, and (v)finally recomposited pixel data sets (recomposited within the vertexand/or fragment shaders of the primary GPU) are displayed as graphicalimages on one or more display devices 106 connected to the primary GPU715 via an analog display interface.

Various Options for Implementing the MMPGRS of the Present Invention

There are numerous options for implementing the various possible designsfor the MMPGRS of the present invention taught herein. Also, as theinventive principles of the MMPGRS can be expressed using software andhardware based system architectures, the possibilities for the MMPGS arevirtually endless.

In FIGS. 8A through 11D1, there is shown a sampling of the illustrativeimplementations that are possible for the diverse MMPGRS designs of thepresent invention disclosed, taught and suggested herein.

FIG. 8A shows an illustrative implementation of the MMPGRS of thepresent invention following the Hub Class of MMPGRS Architecturedescribed in FIG. 7A 1-2. As shown in this MMGPRS implementation, (i)the AMCM and Decomposition Submodule No. 1 are implemented as asoftware-package 701 within host memory space (HMS) of the hostcomputing system, (ii) multiple discrete graphic cards are connected tothe bridge circuit of the host computing system by way of ahardware-based graphics hub chip of the present invention 404″, 402″,403″, 404″, (iii) hardware-based Distribution and Recomposition Modules402″ and 403″ are realized on the hardware-based graphics hub chip ofthe present invention, and (iv) a graphics display device is connectedto the primary GPU.

FIG. 8A 1 shows a first illustrative embodiment of the MMPGRSimplementation of FIG. 8A, wherein a possible packaging of the Hubarchitecture of the present invention employs an assembly comprising aHub-extender card 811 carrying multiple (e.g. dual) graphics cards 812,813 supported on a motherboard 814 within the host computing system.

FIG. 8A 2 shows a second illustrative embodiment of the MMPGRSimplementation of FIG. 8A, wherein a possible packaging of the Hubarchitecture of the present invention employs an external box containinga Hub chip of the present invention mounted on a PC board, that isconnected to the motherboard of the host computing system via a wireharness or the like, and supporting a plurality of graphics cards 813that are connected to the Hub chip.

FIG. 8A 3 shows a third illustrative embodiment of the MMPGRSimplementation of FIG. 8A, wherein a possible packaging of the Hubarchitecture of the present invention employs a graphics hub chip of thepresent invention mounted on the motherboard 814 of the host computingsystem, which supports multiple graphics cards 813 with multiple GPUs.

FIG. 8B shows an illustrative implementation of the MMPGRS of thepresent invention following the Hub+GPU-Recomposition Class of MMPGRSArchitecture described in FIG. 7A 1-2. As shown in this MMPGRSimplementation, (i) the AMCM and Decomposition Submodule No. 1 areimplemented as a software-package 701 within host memory space (HMS) ofthe host computing system, (ii) multiple discrete graphic cards areconnected to a bridge chipset on the host computing system by way of ahardware-based graphics hub chip realizing the Decomposition No. 2Submodule 401″ and the Distribution Module 402″, (iii) the RecompositionModule 403″ is implemented across two or more GPUs 715, 716, and (iv) agraphics display device is connected to the primary GPU.

FIG. 8B 1 shows a first illustrative embodiment of the MMPGRSimplementation of FIG. 8B, wherein a possible packaging of the Hub+GPURecomposition architecture of the present invention employs an assemblycomprising a Hub-extender card 811 carrying multiple (e.g. dual)graphics cards 812, 813 supported on a motherboard 814 within the hostcomputing system.

FIG. 8B 2 shows a second illustrative embodiment of the MMPGRSimplementation of FIG. 8B, wherein a possible packaging of the Hubarchitecture of the present invention employs an external box containinga Hub chip of the present invention mounted on a PC board, that isconnected to the motherboard of the host computing system via a wireharness or the like, and supporting a plurality of graphics cards 813that are connected to the Hub chip.

FIG. 8B 3 shows a third illustrative embodiment of the MMPGRSimplementation of FIG. 8B, wherein a possible packaging of the Hubarchitecture of the present invention employs a graphics hub chip of thepresent invention mounted on the motherboard 814 of the host computingsystem, which supports multiple graphics cards 813 with multiple GPUs.

FIG. 8C shows an illustrative embodiment of the MMPGRS of the presentinvention following the HM Class of MMPGRS Architecture described inFIG. 7A 1-2. As shown in this MMPGRS implementation, (i) the AMCM,Decomposition, Distribution and Recomposition Modules are implemented asa software-package 701 within host memory space (HMS) of the hostcomputing system, (ii) multiple discrete GPUs on one or more graphicscards are connected to the bridge circuit on the host computing system,and (iii) a graphics display device is connected to the primary GPU.

FIG. 8C 1 shows a first illustrative embodiment of the MMPGRSimplementation of FIG. 8C, wherein discrete multiple graphics cards 851,each supporting at least a single GPU, are interfaced with the bridgecircuit chipset of the CPU mother board by way of a PCI-express or likeinterface.

FIG. 8C 2 shows a second illustrative embodiment of the MMPGRSimplementation of FIG. 8C, wherein multiple GPUs are realized on asingle graphics card 852 which is interfaced to bridge circuit on theCPU motherboard by way of a PCI-express or like interface.

FIG. 8C 3 shows a third illustrative embodiment of the MMPGRSimplementation of FIG. 8C, wherein multiple discrete graphics cards 851,each supporting at least a single GPU, are interfaced with the bridgecircuit on a board within an external box 821 that is interface to themotherboard within the host computing system.

FIG. 8D shows an illustrative embodiment of the MMPGRS of the presentinvention following the Hub+GPU-Recomposition Class of MMPGRSArchitecture described in FIG. 7A 1-2. As shown in this MMPGRSimplementation, (i) the AMCM, Decomposition Submodule No. 1 and theDistribution Module are implemented as a software-package 701 withinhost memory space (HMS) of the host computing system, (ii) multiplediscrete GPUs on one or more external graphics cards are connected tothe bridge circuit of the host computing system, (iii) the RecompositionModule 403″ is implemented across two or more GPUs, and (iv) a graphicsdisplay device is connected to the primary GPU.

FIG. 8D 1 shows a first illustrative embodiment of the MMPGRSimplementation of FIG. 8D, wherein discrete multiple graphics cards 851,each supporting at least a single GPU, are interfaced with the bridgecircuit chipset of the CPU motherboard by way of a PCI-express or likeinterface.

FIG. 8D 2 shows a second illustrative embodiment of the MMPGRSimplementation of FIG. 8D, wherein multiple GPUs are realized on asingle graphics card 852 which is interfaced to bridge circuit on theCPU motherboard by way of a PCI-express or like interface.

FIG. 8D 3 shows a third illustrative embodiment of the MMPGRSimplementation of FIG. 8D, wherein multiple discrete graphics cards 851,each supporting at least a single GPU, are interfaced with the bridgecircuit on a board within an external box 821 that is interface to themotherboard within the host computing system.

FIG. 9A shows an illustrative implementation of the MMPGRS of thepresent invention following the Hub Class of MMPGRS Architecturedescribed in FIG. 7A 1-2. As shown in this MMPGRS implementation, (i)the AMCM and Decomposition Submodule No. 1 are implemented as a softwarepackage 711 on the host memory space (PMS), (ii) multiple GPUs (i.e.Primary GPU 715 and Secondary GPUs 716) are assembled on a externalgraphics card 902 which connects the GPUs to the bridge circuit on thehost computing system by way of a hardware-based graphics hub chipimplementing the Decomposition Submodule No. 2 401″, the DistributionModule 402″ and the Recomposition Module 403″, and (iii) a graphicsdisplay device is connected to the primary GPU.

FIG. 9A 1 shows an illustrative embodiment of the MMPGRS of FIG. 9A,wherein multiple GPUs (715, 716) and hardware-based DecompositionSubmodule No. 2 401″, Distribution Module 402″ and the RecompositionModules 403″ are implemented as a graphics hub chip or chipset 401″,402′, 403,″ and 404″ on a single graphics display card 902, which isinterfaced to the bridge circuit on the motherboard 814 within the hostcomputing system.

FIG. 10A shows an illustrative implementation of the MMPGRS of thepresent invention following the Hub Class of MMPGRS Architecturedescribed in FIG. 7A 1-2. As shown in this MMGPRS implementation, (i)the AMCM and Decomposition Submodule No. 1 are implemented as a softwarepackage 711 on the host memory space (HMS), (ii) a single SOC-basedgraphics chip 1001 is mounted on a single graphics card 1002 interfacedwith a bridge circuit on the motherboard 1002, and supporting multipleGPUs (i.e. the primary GPU and secondary GPUs), (iii) hardware-basedDecomposition Submodule No. 2, the Distribution Module and theRecomposition Module are implemented on the SOC-based graphics chip1001, and (iv) a graphics display device is connected to the primaryGPU.

FIG. 10A 1 shows a possible packaging of the SOC-based graphics hub chip(101) depicted in FIG. 10A, wherein multiple GPUs 715, 716 andhardware-based Decomposition Submodule 401″, Distribution Module 402″,and Recomposition Module (4-3″) are realized on a single SOCimplementation 1001 mounted on a single graphics card 1002.

FIG. 10B shows an Illustrative implementation of the MMPGRS of thepresent invention following the Hub+GPU-Recomposition Class of MMPGRSArchitecture described in FIG. 7A 1-2. As shown in this MMPGRSimplementation, (i) the AMCM and Decomposition Submodule No. 1 areimplemented as a software package 711 on the host memory space (HMS),(ii) a single SOC-based graphics chip 1003 is mounted on a singlegraphics card 1002 interfaced with a bridge circuit on the motherboard1002, and supporting multiple GPUs (i.e. the primary GPU and secondaryGPUs), (iii) hardware-based Decomposition Submodule No. 2 and theDistribution Module are implemented on the SOC-based graphics hub chip1001, (iv) the Recomposition Module is implemented across two or moreGPUs 715, 716, and (v) a graphics display device is connected to theprimary GPU by way of a display interface implemented on the SOC-basedgraphics hub chip.

FIG. 10B 1 shows a possible packaging of the SOC-based graphics hub chip101 depicted in FIG. 10B, wherein multiple GPUs 715, 716 andhardware-based Decomposition Submodule 401″, Distribution Module 402′are implemented in single SOC implementation 1003 mounted on a singlegraphics card 1002, with the Recomposition Module (4-3″) beingimplemented across two or more of the GPUs (e.g. on the same piece ofsilicon).

FIG. 10C shows an illustrative implementation of the MMPGRS of thepresent invention following the HMS+GPU-Recomposition Class of MMPGRSArchitecture described in FIG. 7A 1-2. IN this MMPGRS implementation,(i) the AMCM, Decomposition Module and Distribution Module areimplemented as a software package 701 on the host memory space (HMS),(ii) a single multi-GPU chip 1031 is mounted on a single graphics card1002 that is interfaced with a bridge circuit on the motherboard, andsupporting multiple GPUs (i.e. the primary GPU and secondary GPUs),(iii) the Recomposition Module is implemented within two or more GPU,and (iv) a graphics display device is connected to the primary GPU byway of a display interface implemented on the multi-GPU chip.

FIG. 10C 1 shows a possible packaging of the multi-GPU chip (1031)depicted in FIG. 10C, wherein multiple GPUs 715, 716 and RecompositionModule 403″ is implemented across two or more GPUs 715, 716 of amulti-GPU chip (1031).

FIG. 11A shows an illustrative implementation of the MMPGRS followingthe Chipset Class of MMPGRS Architecture described in FIG. 7A 1-2. Inthis MMPGRS implementation, (i) the AMCM and Decomposition Submodule No.1 are realized as a software package 711 within the host memory space(HMS) of the host computing system, (ii) plurality of GPUs 852 on one ormore external graphics cards 851 are connected to the bridge circuit onthe host computing platform, (iii) an integrated graphics device (IGD)1101, supporting hardware-based Decomposition Submodule No. 2, theDistribution Module 402″ and Recomposition Module 403″, are implementedwithin the bridge circuit 1101 on the motherboard 814 of the hostcomputing system, and (iv) a display device is interfaced to the primaryGPU by way of a PCI-express interface or the like.

FIG. 11A 1 shows a first illustrative embodiment of the Chipset MMPGRSimplementation of FIG. 11A, wherein multiple discrete graphics cards851, each supporting at least a single GPU, are interfaced with thebridge circuit on a board within an external box 821 that is interfaceto the motherboard within the host computing system.

FIG. 11A 1 shows a second illustrative embodiment of the Chipset MMPGRSimplementation of FIG. 11A, wherein discrete multiple graphics cards851, each supporting at least a single GPU, are interfaced with thebridge circuit chipset of the CPU motherboard by way of a PCI-express orlike interface.

FIG. 11A 3 shows a third Illustrative embodiment of the Chipset MMPGRSimplementation of FIG. 11A, wherein multiple GPUs are realized on asingle graphics card 852 which is interfaced to bridge circuit on theCPU motherboard by way of a PCI-express or like interface.

FIG. 11B shows an illustrative implementation of the MMPGRS followingthe CPU/GPU Fusion Class of MMPGRS Architecture or Multi-Core ClassMMPGRS Architecture described in FIG. 7A 1-2. As shown in this MMPGRSimplementation, (i) a CPU/GPU fusion-architecture chip or a multi-coreCPU chip is mounted on the motherboard of a host computing system havingmemory and North and South bridge circuits, (ii) AMCM and DecompositionSubmodule No. 1 are realized as a software package 701 within the hostmemory space (HMS) of the host computing system while DecompositionSubmodule No. 2, the Distribution Module and the Recomposition Moduleare realized/implemented on the die of the CPU/GPU fusion-architecturechip or the multi-core CPU chip, and (iii) multiple GPUs on externalgraphic cards or elsewhere, are interfaced to the CPU/GPUfusion-architecture chip or the multi-core CPU chip, by way of aPCI-express or like interface, and (iv) a display device is interfacedto the primary GPU by way of a PCI-express interface or the like.

FIG. 11B 1 shows a first illustrative embodiment of the CPU/GPU Fusionor Multi-Core MMPGRS implementation of FIG. 11B, wherein a CPU/GPUFusion or Multi-Core chip is used to drive an assemble of graphic cardsor GPUs on one or more external graphics cards 851.

FIG. 11B 2 shows a second illustrative embodiment of the Chipset MMPGRSimplementation of FIG. 11B, wherein a CPU/GPU Fusion or Multi-Core chipis used to drive an assemble of GPUs on a single external graphics card852.

FIG. 11B 3 shows a third illustrative embodiment of the Chipset MMPGRSimplementation of FIG. 11B, wherein a CPU/GPU Fusion or Multi-Core chipis used to drive only an assemble of internal GPUs on the CPU/GPU Fusionor Multi-Core chip.

FIG. 11C shows an illustrative implementation of the MMPGRS followingthe Game Console Class of MMPGRS Architecture described in FIG. 7A 1-2.As shown in this MMPGRS implementation, (i) the AMCM 400 andDecomposition Submodule No. 1 401′ are realized as a software packagewithin the host memory space (HMS) of the game console system, (ii) agraphics hub chip 401″, 402″, 403″, 404″, mounted on the PC board of thegame console system, implements the Decomposition Submodule No. 2 401″,the Distribution Module 402′, the Recomposition Module 403′ as well asthe interconnect network (e.g. crossbar switch) 404″, (iii) multipleGPUs on the PC board of the game console system are interfaced toDistribution and Recomposition Modules by way of the interconnectnetwork 404″ within the graphics hub chip, and optionally, theRecomposition Module can be implemented across two or more GPUs 715,716, and (iv) a display device 106 is interfaced to the primary GPU byway of an analog display interface or the like.

FIG. 11C 1 shows an illustrative embodiment of the Game Console MMPGSimplementation of FIG. 11D, showing its controller in combination withits game console unit.

The MMPGRS of the Present Invention Deployed in Client Machines onMulti-User Computer Networks

In the illustrative embodiments described above, the graphics-basedapplications (e.g. games, simulations, business processes, etc.)supporting 3D graphics processes which are rendered using the parallelcomputing principles of the present invention, have been shown as beingsupported on single CPU-based host computing platforms, as well asmulti-core CPU platforms. It is understood, however, that the parallelgraphics rendering processes carried out using the principles of thepresent invention can stem from applications supported on (i) multi-CPUhost computing platforms, as well as (ii) single and multiple CPU basednetwork-based application servers.

In the case of network-based application servers, streams of graphicscommands and data (GCAD) pertaining to the graphics-based application athand can be generated by application server(s) in response to one ormore multiple users (e.g. players) who may be either local or remotewith respect to each other. The application servers would transmitstreams of graphics commands and data to the participants (e.g. users orplayers) of a multi-player game. The client-based computing machine ofeach user would embody one form of the MMPGRS of the present invention,and receive the graphics commands and data streams support theclient-side operations of either (i) a client-server based application(running at the remote application servers), and/or (ii) a Web-basedapplication generated from http (Web) servers interfaced to applicationservers, driven by database servers, as illustrated in FIGS. 12A and12B. In such multi-user computer network environments, the MMPGRS aboardeach client machine on the network would support its parallel graphicsrendering processes, as described in great detail hereinabove, andcomposited images will be displayed on the display device of the clientmachine. Display devices available to the users of a particulargraphics-based application can include LCD panels, plasma displaypanels, LCD or DLP based multi-media projectors and the like.

FIG. 12A shows a first illustrative embodiment of the multi-usercomputer network according to the present invention, comprising aplurality of client machines, wherein one or more client machines embodythe MMPGRS of the present invention designed using the software-basedsystem architecture of FIG. 7A. In FIG. 12B, a second illustrativeembodiment of the multi-user computer network of the present invention,is shown comprising a plurality of client machines, wherein one or moreclient machines embody the MMPGRS of the present invention designedusing the hardware-based system architecture of FIG. 7B. In eithernetwork design, the application server(s), driven by one or moredatabase servers (RDBMS) on the network, and typically supported by acluster of communication servers (e.g. running http), respond touser-system interaction input data streams that have been transmittedfrom one or more network users on the network. Notably, these user (e.g.gamers or players) might be local each other as over a LAN, or be remoteto each other as over a WAN or the Internet infrastructure. In responseto such user-system interaction, as well as Application profilingcarried out in accordance with the principles of the present invention,the MMPGRs aboard each client machine will automatically control, inreal-time, the mode of parallel graphics rendering supported by theclient machine, in order to optimize the graphics performance of theclient machine.

Using A Central Application Profile Database Server System toAutomatically Update Over the Internet, Graphic Application Profiles(GAPs) Within The MMPGRS of Client Machines

It is with the scope and spirit of the present invention to ensure thateach MMPGRS is optimally programmed at all possible times so that itquickly and continuously offers users high graphics performance throughits adaptive multi-modal parallel graphics operation. One way to helpcarry out this objective is to set up a Central Application/SceneProfile Database Server System on the Internet, as shown in FIGS. 12Aand 12B, and support the various Internet-based application registrationand profile management and delivery services, as described hereinbelow.

As shown in FIGS. 12A and 12B, the Central Application/Scene ProfileDatabase Server System of the illustrative embodiment comprises acluster of Web (http) servers, interfaced with a cluster of applicationservers, which in turn are interfaced with one or more database servers(supporting RDBMS software), well known in the art. The CentralApplication/Scene Profile Database Server System would support aWeb-based Game Application Registration and Profile ManagementApplication, providing a number of Web-based services, including:

(1) the registration of Game Application Developers within the RDBMS ofthe Server;

(2) the registration of game applications with the RDBMS of the CentralApplication/Scene Profile Database Server System, by registered gameapplication developers;

(3) registration of each MMPGRS deployed on a client machine or serversystem having Internet-connectivity, and requesting subscription toperiodic/automatic Graphic Application Profile (GAP)

Updates (downloaded to the MMPGRS over the Internet) from the CentralApplication Profile Database Server System, and

(4) registration of each deployed MMPGRS requesting the periodicuploading of its Game Application Profiles (GAPS)—stored inApplication/Scene Profile Database 405 and Historical Repository 404—tothe Central Application/Scene Profile Database Server System for thepurpose of automated analysis and processing so as to formulate “expert”Game Application Profiles (GAPs) that have been based on robustuser-experience and which are optimized for particular client machineconfigurations.

Preferably, the Web-based Game Application Registration and ProfileManagement Application of the present invention would be designed (usingUML techniques) and implemented (using Java or C+) so as to provide anindustrial-strength system capable of serving potentially millions ofclient machines embodying the MMPGRS of the present invention.

Using the Central Application/Scene Profile Database Server System ofthe present invention, it is now possible to automatically andperiodically upload, over the Internet, Graphic Application Profiles(GAPs) within the Application/Scene Profile Database 405 of the MMPGRSof registered client machines. By doing so, graphic application users(e.g. gamers) can immediately enjoy high performance graphics on thedisplay devices of their client machines, without having to develop arobust behavioral profile based on many hours of actual user-systeminteraction, but rather, automatically periodically uploading in theirMMPGRSs, “expert” GAPs generated by the Central Application/SceneProfile Database Server System by analyzing the GAPs of thousands ofgame application users connected to the Internet.

For MMPGRS users subscribing to this Automatic GAP Management Service,supported by the Central Application/Scene Profile Database ServerSystem of the present invention, it is understood that such MMPGRSswould use a different type of Application Profiling and Analysis thanthat disclosed in FIGS. 5C1 and 5C2.

For Automatic GAP Management Service subscribers, the MMPGRS wouldpreferably run an application profiling and analysis algorithm that usesthe most recently downloaded expert GAP loaded into its AMCM, and thenallow system-user interaction, user behavior, and applicationperformance to modify and improve the expert GAP profile over time untilthe next automated update occurs.

Alternatively, the Application Profiling and Analysis Module in eachMMGPRS subscribing to the Automatic GAP Management Service, will bedesigned to that it modifies and improves the downloaded expert GAPwithin particularly set limits and constraints, and according toparticular criteria, so that the expert GAP is allowed to evolve in anoptimal manner, without performance regression.

For users, not subscribing to the Automatic GAP Management Service,Application Profiling and Analysis will occur in their MMPGRSs accordingto general processes described in FIGS. 5C1 and 5C2.

Variations of the Present Invention Which Readily Come to Mind in Viewof the Present Invention Disclosure

While the illustrative embodiments of the present invention have beendescribed in connection with various PC-based computing systemapplications and video game consoles and systems, it is understood thatthat multi-modal parallel graphics rendering subsystems, systems andrendering processes of the present invention can also be used in mobilecomputing devices, e-commerce and POS displays and the like.

While Applicants have disclosed such subsystems, systems and methods inconnection with Object, Image and Time Division methods beingautomatically instantiated in response to the graphical computing needsof the application(s) running on the host computing system at anyinstant in time, it is understood, however, that the MMPGRS of thepresent invention can be programmed with other modes of 3D graphicsrendering (beyond traditional Object, Image and Time Division Modes),and that these new and/or extended mode of parallel operation can bebased on novel ways of dividing and/or quantizing: (i) objects and/orscenery being graphically rendered; (ii) the graphical display screen(on which graphical images of the rendered object/scenery areprojected); (iii) temporal aspects of the graphical rendering process;(iv) the illumination sources used during the graphical renderingprocess using parallel computational operations; as well as (v) varioushybrid combinations of these components of the 3D graphical renderingprocess.

It is understood that the multi-modal parallel graphics renderingtechnology employed in computer graphics systems of the illustrativeembodiments may be modified in a variety of ways which will becomereadily apparent to those skilled in the art of having the benefit ofthe novel teachings disclosed herein. All such modifications andvariations of the illustrative embodiments thereof shall be deemed to bewithin the scope and spirit of the present invention as defined by theClaims to Invention appended hereto.

1. A multi-mode parallel graphics rendering system (MMPGRS) embodiedwithin a host computing system having (i) host memory space (HMS) forstoring one or more graphics-based applications and a graphics libraryfor generating graphics commands and data (GCAD) during the run-time ofthe graphics-based application, (ii) one or more CPUs for executing saidgraphics-based applications, and (iii) a display device for displayingimages containing graphics during the execution of said graphics-basedapplications, said MMPGRS comprising: a multi-mode parallel graphicsrendering subsystem supporting multiple modes of parallel operationselected from the group consisting of object division, image division,and time division; a plurality of graphic processing pipelines (GPPLs)supporting a parallel graphics rendering process that employs one ormore of said object division, image division and/or time division modesof parallel operation in order to execute graphic commands and processgraphics data, and render pixel-composited images containing graphicsfor display on a display device during the run-time of saidgraphics-based application; and an automatic mode control module forautomatically controlling the mode of parallel operation of said MMPGRSduring the run-time of said graphics-based application.
 2. The MMPGRS ofclaim 1, wherein said automatic mode control module employs theprofiling of scenes in said graphics-based application.
 3. The MMPGRS ofclaim 1, wherein said automatic mode control module employs theprofiling of scenes in said graphics-based application, on an imageframe by image frame basis.
 4. The MMPGRS of claim 2, wherein saidprofiling of scenes in said graphics-based application, is carried outin real-time during run-time of said graphics-based application.
 5. TheMMPGRS of claim 4, wherein said profiling of scenes in saidgraphics-based application, is carried out in real-time, during run-timeof said graphics-based application, on an image frame by image framebasis.
 6. The MMPGRS of claim 4, wherein said real-time profiling ofscenes in said graphics-based application involves (i) collecting andanalyzing performance data associated with said MMPGRS and said hostcomputing system, during application run-time, (ii) constructing sceneprofiles for the image frames associated with particular scenes in saidparticular graphics-based application, and (iii) maintaining said sceneprofiles in an application/scene profile database that is accessible tosaid automatic mode control module during run-time, so that during therun-time of said graphics-based application, said automatic mode controlmodule can access and use said scene profiles maintained in saidapplication/scene profile database and determine how to dynamicallycontrol the modes of parallel operation of said MMPGRS to optimizesystem performance.
 7. The MMPGRS of claim 2, wherein said automaticmode control module employs real-time detection of scene profile indicesdirectly programmed within pre-profiled scenes of said graphics-basedapplication; wherein said pre-profiled scenes are analyzed prior torun-time, and indexed with said scene profile indices; and wherein andmode control parameters (MCPS) corresponding to said scene profileindices, are stored within a application/scene profile databaseaccessible to said automatic mode control module during applicationrun-time.
 8. The MMPGRS of claim 6, wherein during run-time, saidautomatic mode control module automatically detects said scene profileindices and uses said detected said scene profile indices to accesscorresponding MCPs from said application/scene profile database so as todetermine how to dynamically control the modes of parallel operation ofsaid MMPGRS to optimize system performance.
 9. The MMPGRS of claim 2,wherein said automatic mode control module employs real-time detectionof mode control commands (MCCs) directly programmed within pre-profiledscenes of said graphics-based application; wherein said pre-profiledscenes are analyzed prior to run-time, and said MCCs are directlyprogrammed within the individual image frames of each scene; and whereinduring run-time, said automatic mode control module automaticallydetects said MCCs along the graphics command and data stream, and usessaid MCCs so as to determine how to dynamically control the modes ofparallel operation of said MMPGRS to optimize system performance. 10.The MMPGRS of claim 1, wherein said automatic mode control moduleemploys a user interaction detection (UID) mechanism for real-timedetection of the user's interaction with said host computing system. 11.The MMPGRS of claim 10, wherein, in conjunction with said sceneprofiling, said automatic mode control module also uses said UIDmechanism to determine how to dynamically control the modes of paralleloperation of the MMPGRS to optimize system performance, at any instancein time during run-time of said graphics-based application.
 12. TheMMPGRS of claim 11, wherein said UID mechanism analyzes user-systeminteraction data generated within said host computing system.
 13. TheMMPGRS of claim 11, wherein said user-system interaction data representsan event selected from the group consisting of mouse device movement,keyboard depressions, head movement, voice commands, eye movement, feetmovement, and graphics-based application (e.g. game) updates originatedover the LAN, WAN or Internet (WWW).
 14. The MMPGRS of claim 1, whereinonly one of said GPPLs is designated as the primary GPPL and isresponsible for driving said display unit with a final pixel imagecomposited within a frame buffer (FB) maintained by said primary GPPL,and all other GPPLs function as secondary GPPLs, supporting the pixelimage recompositing process.
 15. The MMPGRS of claim 14, wherein eachsaid GPPL is a GPU-based GPPL which comprises: (i) a video memorystructure supporting a frame buffer including stencil, depth and colorbuffers, and (ii) a graphics processing unit (GPU) supporting (1) ageometry subsystem having an input assembler, a vertex shader and ageometry shader, (2) a rasterizer, and (3) a pixel subsystem including apixel shader receiving pixel data from said frame buffer and rasteroperators operating on pixel data in said frame buffer.
 16. The MMPGRSof claim 1, wherein each said GPPL is a CPU-based GPPL which comprises:(i) a video memory structure supporting a frame buffer includingstencil, depth and color buffers; and (ii) a graphics processingpipeline realized by one cell of a multi-core CPU chip, consisting of aplurality of processors.
 17. The MMPGRS of claim 16, wherein saidplurality of processors comprises a plurality of in-order SIMDprocessors.
 18. The MMPGRS of claim 16, wherein said CPU-based GPPLfurther comprises: (iii) a texture sampler for loading texture maps frommemory, filtering said texture maps for level-of-detail, and feedingsaid texture maps to the pixel processing portion of said CPU-basedGPPL.
 19. The MMPGRS of claim 1, wherein said MMPGRS has system statesselected from the group consisting of (i) parallel graphics renderingstates, (ii) a non-parallel graphics rendering state, (iii) anapplication identification state, and (iv) a trial & error cycle state.20. The MMPGRS of claim 18, wherein said parallel graphics renderingstates include: (i) an image division state attained when said MMPGRS isoperating in said image division mode; (i) an object division stateattained when said MMPGRS is operating in said object division mode; and(iii) a time division state attained when said MMPGRS is operating insaid time division mode.
 21. The MMPGRS of claim 19, wherein saidnon-parallel graphics rendering state is attained only when a singleGPPL is operational during the graphics rendering process.
 22. TheMMPGRS of claim 1, wherein said display device is a device selected fromthe group consisting of an flat-type display panel, a projection-typedisplay panel, and other image display devices.
 23. The MMPGRS of claim1, wherein said host computing system is a machine selected from thegroup consisting of a PC-level computer, information server, laptop,game console system, portable computing system, and anycomputational-based machine supporting the real-time generation anddisplay of 3D graphics.